Reflex

Reflex is an autonomous ML research agent that runs the entire model-improvement loop by itself. Writing code, training, evaluating, and iterating until it converges on a optimal solution.

Comment

Project Description

The problem statement asks for an autonomous ML research agent: a system that itself reads the problem, proposes hypotheses, implements and runs them, evaluates results, and decides what to try next — with minimal human intervention — converging per a fixed rule (ε=0.002 improvement over the last N=3 iterations, or a 50-iteration / 6-hour cap).

  1. agent_loop.py — the autonomous agent itself. A Python driver that calls Claude headlessly (via the local Claude Code binary in --print mode — genuine, separately metered API calls) once per iteration to propose a configuration from a fixed action space (baseline.py's existing --loss, --wt_target, --k, --lr, --aux_weight, --dns_n, --adt_beta flags — no new code is written by the agent). Each proposal is run as a real training job, its validation/test metrics are read back, and the competition's own convergence rule is checked in Python code — not delegated to the model's judgment.

The prompt also grounds every proposal in a neutral dataset-facts block (catalog size, interaction density, label-threshold behavior — numbers, not conclusions) and a method-reference sheet naming each loss's origin paper and the assumption it relies on, with a required mechanism_basis field forcing every choice to cite a specific fact and assumption rather than restate what a loss generically does.

Four runs were needed to reach the submitted result, each fixing a specific, diagnosed problem in the one before it — the full story is in RUN_AND_ITERATION_LOG.md. Run 1 (Haiku, no dataset context) produced generic, docstring-level hypotheses. Run 2 (Sonnet, grounded) reasoned dramatically better but converged to a statistically identical score, because it anchored on k=32 in iteration 1 and never revisited it — a hyperparameter-anchoring bug diagnosed via a stability check on an unrelated experiment. Run 3 fixed that (a parameter-coverage summary + required dimension_check field, plus a shortlist-confirmation step — the harness runs the full 5-seed confirmation on the top 3 distinct candidates, not just the nominal single-seed best, since the search's top candidates had started clustering inside this project's own single-seed noise floor) and converged in 8 iterations to test primary 0.6016, statistically matching the hand-driven track's own long-established default.

Run 4 (submitted) came from a subsequent, deliberate full re-read of the problem statement against the implementation — not prompted by an observed failure — which found three real compliance gaps in Run 3's harness: (a) every iteration's log entry displayed the candidate's hidden-test score, and that log is exactly what the next proposal call reads back as history, meaning the model's own reasoning could see test feedback across iterations, contrary to "develops using only train + public validation feedback"; (b) the spec fixes the convergence rule (ε=0.002, N=3) under "Allowed Assumptions" — an organizer-given constant — but Run 3 had raised N to 5 to work around the anchoring problem, the wrong fix for a correctly-diagnosed issue; (c) Task Requirement 1 ("reproduce the official baseline... confirm it reaches the reported validation score") was never an explicit, autonomous, verified step, and the codebase had actually drifted to where it could no longer reproduce the literal official config at all (two temporal features had been permanently added to the default 5-field encoding). All three fixed: test scores are recorded only for the final report, never shown to the proposing LLM mid-run; patience_n reverted to the fixed N=3 (anchoring stays fixed via the prompting change alone); and agent_loop.py now runs a code-verified baseline-reproduction check before iteration 1 (confirmed: single-seed valid 0.6015 vs. published 0.6016). Re-run clean under all three fixes: converged in 7 iterations, $0.48, landing on pairwise_watchtime (k=16, lr=0.001, aux_weight=0.5): test GAUC 0.6700 / nDCG@5 0.5332, a +0.0070 score_dataset over baseline — statistically identical to Run 3's result despite the different specific config, and again statistically indistinguishable from the hand-driven default.

The action space was deliberately kept narrow — CLI-flag selection, not free-form code generation — as a considered reliability trade-off: a tight space that reliably runs to convergence beats an ambitious one that breaks mid-loop, and it isolates what's being measured (autonomous propose→test→evaluate→decide iteration) from open-ended architecture search. Section 2.2 of the problem statement states that "writing the code for each stage is part of the agent's job" — this design deliberately does not do that, which is the direct motivation for part 3 below.

  1. agent_loop_codegen.py — a second, more spec-literal agent that actually writes code. Same discipline (headless Claude, decisions made in Python code, ε=0.002/N=3 convergence, one manual intervention), but each iteration invokes Claude with real Read/Write/Edit/Bash tool access — not a JSON-schema proposal — inside an isolated git worktree, and it genuinely writes and edits agent_pipeline.py: feature engineering, model architecture, training strategy are all authored by the agent, not selected from a fixed menu. evaluate.py and data_guard.py (a new protected shim) are read-only interfaces the agent must call, not edit — any diff touching them is a caught guardrail violation (reverted, logged as an error/recovery event), matching the two things the spec itself fixes under Allowed Assumptions (the split and the scoring script). Test-blindness is architectural, not prompted: data_guard.py physically omits the hidden test split from what it returns unless the harness's own once-only final confirmation sets an environment variable the agent's code cannot trigger — verified directly.

Two isolation bugs were found and fixed before trusting any result from this agent, both the hard way. A git worktree checks out the entire repository by default, so an early run had full Read access to RUN_LOG.md and every other file documenting the hand-driven track's own conclusions — caught when that run cited RUN_LOG.md by name as evidence for a hypothesis, which would have made "independent discovery" meaningless. Fixed with git sparse-checkout and an explicit allowlist (data.py, evaluate.py, data_guard.py, temporal_features.py, submit.py) — the exclusion is physical, not a prompt instruction the agent's own Read tool could route around. A second run then cited baseline.py's inline comments (which state hand-driven verdicts directly, some also citing RUN_LOG.md) — baseline.py was promptly removed from the allowlist too, even though it's a legitimate reference implementation, because the specific copy in this repo is not a safe one to hand a "blind" agent. (One smaller leak remains and is disclosed rather than patched over: data.py/temporal_features.py are required, protected interfaces whose own inline comments briefly mention that two engineered fields were "verified in RUN_LOG.md to have stable gains" — a real, small residual leak we chose not to fix by forking the canonical data pipeline.)

The clean run's result exceeded every other number in this project. Converged in 10 iterations (1 recovered proposal-call failure), $5.01, reproducing the official baseline from scratch on iteration 1, then independently rediscovering pointwise→BPR and the watch-time auxiliary task — the same two interventions both the CLI-flag agent and the hand-driven track found, at closely matching scores, on completely independent code. Its most consequential move was novel, not a rediscovery: iteration 6 engineered a general per-user prior-interaction-count feature — no prior part of this project (hand-driven or either agent) had built this specific signal, only narrower video- and author-specific analogues — reasoning directly from the phrase "prior interaction counts" in its own method-reference sheet. That single feature drove the largest single-iteration jump in the whole run (0.6068 → 0.6101 valid). Two further hypotheses in the same family (an item-side popularity analogue, an is_click multi-task head) were tested and correctly identified as null, with the codegen agent's own diffs cleanly reverted. Final 5-seed confirmation: valid primary 0.6099 ± 0.0002, test primary 0.6038 ± 0.0003 — a result that exceeds not only the submitted agent_loop.py result (0.6016) but the hand-driven track's own best ensemble (0.6034, below), on a single self-written model with no ensembling at all. Submission: submission_pure_codegen.csv, generated from the winning pipeline (agent_pipeline_final.py) and format-validated. Full transcript in AGENT_LOG_CODEGEN.md.

This specific 0.6038 is not the agent's typical output — checked directly via 4 more independent runs, not assumed. Test primary across all 5: mean 0.6016, σ≈0.0013, range 0.0039 — statistically the same average as the CLI-flag agent's own 4-run mean (0.6013). Run 1 is the top of that range, and separately the validation-best of all 9 runs across both agents, which is the competition's own selection rule. Root cause of the variance (visible in the logs): the agent evaluates each method through its own self-written implementation, so one weak implementation early can make a genuinely good method look bad and foreclose a productive branch for the rest of the run. agent_loop_codegen.py gained a --restarts flag as a direct fix — K independent trajectories, one validation-best selection across all of them, keeping manual intervention at exactly 1. Full detail in FINAL_SUBMISSION.md.

  1. A separate, hand-driven research track (RUN_LOG.md, baseline.py, ablation_*.py, sequence_model.py, deepfm_model.py, finalmlp_model.py, lightgcn_model.py, the ensembling scripts, and the bonus-benchmark runners), where a human (using Claude Code interactively as a tool, not as the autonomous loop) explored a much wider space — architectures (DIN, BST, DeepFM, FinalMLP, LightGCN), training strategies (DNS, ADT, temporal reweighting), auxiliary objectives (CWM watch-time, RAD quantile targets), and four ensembling strategies. This reached test primary 0.6034 (heterogeneous ensemble of BST + two FM variants) over four days of literature-guided search — and produced AGENT_VS_MANUAL.md, an honest comparison against the CLI-flag agent. agent_loop_codegen.py's single self-written model exceeding this (0.6038) is itself a notable finding, discussed on its own terms rather than folded into this comparison, since it came from a materially different kind of agent.

We consider the coexistence of all three parts a feature, not a compromise: the hand-driven work establishes what headroom actually exists and where the ceiling sits (useful ground truth for judging whether either autonomous loop is finding real signal or noise), while agent_loop.py and agent_loop_codegen.py are the actual answer to the problem statement's core ask. agent_loop_codegen.py's result is the officially designated submission: it scores higher (0.6038 vs. 0.6016 test primary) and more literally satisfies the problem statement's own framing of the task (it writes the code for each pipeline stage, rather than selecting among a fixed menu of pre-written flags). agent_loop.py's result is kept alongside it as a fully valid, four-times- validated, ~10x-cheaper alternative — the trade-off behind designating the newer, higher-scoring, less-tested design as primary is discussed explicitly in FINAL_SUBMISSION.md, not glossed over.

Development tools

  • Claude Code (Anthropic) — as the interactive tool driving the hand-built research track; as the invoked-headlessly proposal LLM inside agent_loop.py; and, with real Read/Write/Edit/Bash tool access inside an isolated git worktree, as the code-writing agent inside agent_loop_codegen.py — the submitted result.
  • Standard shell/Python tooling on macOS (local development) and a SLURM GPU cluster (for BST and the KuaiRand-27K bonus run).

APIs used

  • Claude (via the local Claude Code CLI binary, headless --print mode) — the only API call the autonomous agent makes. Model used for the submitted run's proposal calls: Sonnet (upgraded from an earlier Haiku-based run specifically to test whether stronger reasoning improves proposal quality — see RUN_AND_ITERATION_LOG.md for that comparison). No other external API was used (no OpenAI, no Google, etc.) and no data outside the officially provided KuaiRand files was used.

Libraries and frameworks

  • NumPy — the entire core baseline (Factorization Machine, all loss variants, BPR sampling, evaluation) is hand-implemented in pure NumPy, matching the starter kit's original numpy-only constraint.
  • PyTorch — added later, isolated to sequence/graph/deep architectures where hand-deriving backprop stopped being worthwhile: sequence_model.py (DIN, BST), deepfm_model.py, finalmlp_model.py, lightgcn_model.py, and bonus_fm_torch.py (a sparse-embedding FM for the KuaiRand-27K bonus benchmark, needed because dense Adam updates over a ~41M-row embedding table are computationally infeasible regardless of hardware).
  • No pandas, scikit-learn, or other ML framework — kept intentionally minimal.

Datasets and assets

  • KuaiRand-Pure (required benchmark) — the official starter-kit dataset, including the standard interaction logs and, for one ablation, the previously-unused official user_features_pure.csv and video_features_statistic_pure.csv side-information files.
  • KuaiRand-1K and KuaiRand-27K (bonus benchmarks) — same task and metrics as Pure, run via a memory-efficient columnar loader built specifically because the starter kit's list-of-tuples approach doesn't scale to these sizes on commodity hardware.
  • No external or synthetic data of any kind was used, per the competition's rules.

Built With

Share this project:

Updates