Inspiration
Recommender-system research is a slow loop: form a hypothesis, implement it, train, evaluate, inspect evidence, and decide what to try next. TikTok TechJam Track 2 asks for an autonomous ML research agent, so we built a system that automates the research loop rather than simply searching hyperparameters.
What it does
EvoResearch combines an AI research agent with genetic-algorithm-inspired evolutionary search.
The AI research agent is responsible for semantic reasoning. It reads prior experiment evidence, forms explicit hypotheses, writes candidate Python implementations, reflects on failures, and proposes semantic mutations and crossovers.
A deterministic Evolution Controller owns search pressure: population management, fitness, parent selection, elitism, diversity, lineage, and budgets. The AI decides what scientific idea to try; the controller decides what survives and where the search goes next.
Our live runs used Gemini 3.6 Flash as the AI provider after Gemini 3.7 Flash was temporarily capacity-blocked, but the research architecture itself is provider-agnostic.
Architecture
Research State -> AI Research Agent -> generated candidate -> Experiment Runner -> official KuaiRand evaluator -> Registry -> Evolution Controller -> next Research State
Candidate experiments are evaluated through the organizer's unchanged evaluate.py. Autonomous research uses the validation split only. The test split requires explicit opt-in and was used only once after the final candidate had been frozen.
Evolutionary search
This is not brute-force AutoML. Mutation and crossover are semantic operations proposed by the AI agent, while selection and survival are deterministic.
Compared with a sequential research agent, the evolutionary system keeps multiple surviving branches, preserves elites after failed children, suppresses duplicates, and can combine distinct successful mechanisms through crossover.
Auditing our own search
A late independent audit found that our first search space had three reachability defects:
- Regularization and the training objective were effectively unreachable.
- A residual-blending strategy could not score below its parent, creating a degenerate evolutionary path.
- Missing diversity metadata had silently weakened duplicate suppression and crossover.
We fixed the search affordances, not the answer. Tests ensure measured values from the audit do not leak into the agent's research state. The next autonomous sprint independently explored a within-user listwise objective, tier-adaptive regularization, and additional model variations.
Results
Validation primary metric:
| Method | New evaluations | Best primary |
|---|---|---|
| Reproduced FM baseline | 0 | 0.6014688 |
| Matched sequential search | 6 | 0.6021109 |
| Matched evolutionary search | 6 | 0.6023186 |
| Post-audit evolutionary sprint | 7 | 0.6029037 |
With the same priors and six new evaluations, sequential search did not improve beyond its starting elite, while evolutionary search reached 0.6023186. After expanding the reachable research space, a second autonomous sprint reached 0.6029037, a +0.0014350 validation improvement over the reproduced FM baseline.
A paired user bootstrap against the FM root gave P(delta > 0) = 0.990 with a 95% interval excluding zero. We do not claim statistical significance over the immediately previous candidate.
Scientific integrity
Selection was closed on validation before test evaluation. The frozen candidate then ran on the test split once.
The additional validation gain did not transfer: the final two candidates were statistically indistinguishable on test, with P(delta > 0) = 0.515. We kept the validation-selected candidate rather than switching after seeing test performance, because doing so would be test-driven model selection.
The runner also performs parent-side integrity checks around every candidate attempt. Protected evaluator, source, starter, and dataset assets are hashed before and after execution; tampered attempts are invalidated before their scores can be accepted. Candidate subprocesses receive a minimal allowlisted environment without provider credentials. These controls protect result integrity, while we explicitly do not claim OS-level sandboxing of generated code.
Autonomy and feasibility
The final autonomous sprint:
- stopped itself on convergence after 7 of 8 evaluations
- took about 46 minutes
- used 0 GPU-hours
- required 0 manual candidate interventions
- produced a final model that was about 22.7% faster on validation than the candidate it replaced on the same harness
The competition controller can enforce a 50-evaluation / 6-hour envelope, epsilon-based convergence, and patience limits, but we intentionally did not burn the full budget once the search had converged.
Reproducibility
The public repository contains the frozen final scorer, experiment configs, iteration evidence, lineage, benchmarks, paired-bootstrap analysis, testing instructions, and free FakeProvider tests. Live AI runs require an API key; reproducing the frozen final candidate does not.
Built for
TikTok TechJam 2026 - Track 2: Autonomous Machine Learning Research Agent for Recommender Systems