About the Project

Why we built Bridges

Improving a recommendation system involves more than training a model repeatedly.

A researcher first needs to understand where the current model is making mistakes. From those errors, they can form a hypothesis, design an experiment, measure the result, and decide whether the evidence is strong enough to justify a change. Sometimes the more useful result is finding that an explanation was wrong.

For TikTok TechJam 2026 Track 2, we built Bridges, an autonomous ML research agent designed around this process.

Rather than giving the agent a fixed list of model improvements to try, we built a research loop around measured evidence. The agent examines the current recommendation model, identifies weaknesses, proposes experiments, runs them through a controlled evaluation system, and records what it learns.

One rule became especially important: a better score is not automatically a correct explanation. When the agent attributes an improvement to a particular mechanism, that explanation can be tested separately before it is accepted.

How we built it

Bridges combines an autonomous research loop with a lightweight recommendation model.

Each research iteration follows a defined process:

Observe. Analyze validation errors from the current best model and find areas where it still performs poorly.
Prioritize. Rank possible research directions based on their expected value relative to experiment time.
Experiment. Implement and evaluate one hypothesis using a shared experiment framework.
Validate. Decide whether to keep the change using validation results, not the final test score.
Challenge the explanation. If the improvement is linked to a proposed mechanism, run a control designed to test that explanation.
Remember. Store successful ideas, rejected hypotheses, controls, and previous findings so later iterations can build on earlier evidence.

Claude Code provides the language-model reasoning used within this research loop. The surrounding system controls experiment execution, validation rules, structured research memory, promotion requirements, and falsification checks.

For the recommendation model itself, we used a Factorization Machine and added lightweight signals describing recent user behavior. In simple terms, the model considers both what a person generally likes and what they appear interested in recently.

All history-based features use information available before the recommendation being scored.

Our main result

On the required KuaiRand-Pure benchmark, the official baseline has a primary score of 0.5946. Our final model reaches 0.6143.

Metric	Official baseline	Bridges	Improvement
GAUC	0.6610	0.6857	+0.0247
nDCG@5	0.5282	0.5429	+0.0147
Primary score	0.5946	0.6143	+0.0197

The final model is a five-run committee. Training and evaluation run on CPU with NumPy. No GPU was used.

What the model learned

A useful pattern came from recent user behavior.

A person's long-term preferences do not necessarily describe what they want at every moment. Someone may normally interact with one type of content but spend the current session interested in something different.

Our model therefore keeps a small amount of prior behavioral information, including recent interaction outcomes, author and tag history, timing information, and prior experience with the current recommendation area.

The improvement alone, however, did not tell us whether recent behavior was actually responsible. So we tried to break that explanation.

What happened when we scrambled time?

Our explanation was that behavioral history helps because the model knows what happened and when it happened.

We tested that explanation by scrambling when the behavioral information appeared within each user's history. The same type of information remained available, but it was attached to the wrong moments.

Experiment	Test primary
Behavior attached to the correct moment	0.61164
Remove sequence features	0.59808
Attach behavior to the wrong moments	0.59872
Replace it with matched random information	0.59876

About 95% of the measured sequence-feature improvement disappeared when we broke the timing.

Incorrectly timed behavior also performed almost exactly like matched random information. This supports the interpretation that correctly timed recent behavior accounts for most of the measured gain from these features.

An improvement found through the research loop

The autonomous research process also produced one of the features used in our final model.

The agent analyzed errors from the existing champion and found a weak validation area. From that evidence, it proposed measuring how much prior experience a user had with the recommendation area currently being viewed.

That experiment produced the feature we call tab_n.

After the required validation checks, it helped move the previous champion from about 0.6116 to the final 0.6143 score.

But there was a catch. Our first explanation for the feature was incomplete.

A later control indicated that part of its effect depended on genuine familiarity with the recommendation area, while another part came from the feature's broader counting structure. We kept the feature because the improvement passed our validation rule, but revised the explanation to match the control.

That result changed how we thought about the research agent. Finding a useful feature and correctly explaining why that feature works are separate tasks.

Testing the same approach at a larger scale

We also tested whether the finished recipe would transfer to KuaiRand-1K.

We kept the model recipe fixed rather than changing its settings specifically to improve the 1K result.

KuaiRand-1K	Primary score
Reproduced baseline	0.6293
Bridges	0.6931
Improvement	+0.0637

The improvement is more than three times the margin measured on KuaiRand-Pure.

There is a useful difference between the datasets. KuaiRand-Pure contains roughly 53 logged interactions per user, while KuaiRand-1K contains about 11,700.

Our model relies partly on recent behavioral history, so deeper histories provide more information for these features. The larger improvement is consistent with that explanation. It is not a controlled test of history depth alone, however, because other differences between the datasets may also contribute.

Challenges we faced

Preventing lucky test results from becoming discoveries

Repeated experimentation creates a risk of choosing a model because its final test score happens to look good.

We therefore separate model selection from final test reporting. Promotion decisions use validation performance only. Test scores are recorded, but they do not decide whether an experiment becomes the new model.

Some configurations with attractive test scores were rejected because their validation results did not justify promotion.

Separating an improvement from its explanation

A feature can improve predictions even when our explanation for that improvement is wrong.

We added falsification controls for this reason. A proposed temporal mechanism can be tested by scrambling time. Other mechanisms can be compared with matched random controls.

A failed control does not disappear from the research record. It changes what the system is allowed to claim.

Keeping experiments reproducible

Autonomous experimentation becomes difficult to audit if every experiment follows a different procedure.

We built a common experiment harness that handles repeated seeds, validation gating, and intent-first logging. The hypothesis is recorded before training, then the result is attached afterwards.

Scaling without a GPU

KuaiRand-1K contains about 11.7 million rows and millions of video IDs.

We adapted feature construction and model training so the transfer experiment could still run with NumPy on a laptop CPU. This included more memory-efficient feature construction and sparse parameter updates.

What we learned

The strongest modeling lesson was fairly simple: recent behavior matters much more when it is attached to the correct moment. Scrambling time removed about 95% of the measured benefit.

The research-system lesson was different.

An autonomous researcher needs to do more than generate ideas. It needs a way to decide which idea deserves testing, reject changes that do not clear the evidence threshold, remember previous results, and challenge explanations that appear convincing after a successful experiment.

Our experiments also showed why failed hypotheses should remain visible. The tab_n control did not erase the improvement. Instead, it forced us to narrow our explanation of why the improvement occurred.

For this project, the distinction between finding a result and earning the right to explain it became one of the most useful parts of the research process.