EpsilonRec’s autonomous pipeline: process requirements, clean data, train models, and revise experiments without human input..
Dashboard view of EpsilonRec turning a recommender research brief into structured ML requirements and experiment actions.
1. Benchmark results from KuaiRand-Pure: EpsilonRec compares candidate recommenders using GAUC and nDCG@5 metrics.

Inspiration

Most recommender-system projects start with a single question: “Which model should we train?”
But in real ML research, the harder question is: “How do we keep improving when most ideas fail?”

That inspired us to build EpsilonRec.

Instead of creating one static recommender model, we wanted to build an autonomous research agent that behaves more like an ML researcher. It should generate hypotheses, train candidate recommenders, evaluate them honestly, reject weak ideas, and decide what to try next.

For the TikTok TechJam KuaiRand-Pure recommendation challenge, this matters because short-video recommendation is not just about predicting clicks. A useful system must handle large-scale user-video interactions, compare candidates with real metrics, and improve through repeated experimentation.

What It Does

EpsilonRec is an autonomous ML research framework for KuaiRand-Pure recommendation experiments.

It does not rely on one “magic” model. Instead, it runs a structured research loop:

Understand the recommendation task and available signals.
Generate candidate modeling ideas.
Train recommender variants.
Evaluate each candidate using benchmark metrics like GAUC and nDCG@5.
Compare results against a baseline.
Accept only candidates that improve validation performance.
Log failures, decisions, and next experiments.

The key idea is that EpsilonRec treats recommendation development as an iterative research process, not a one-shot prediction task.

How We Built It

We designed EpsilonRec around a multi-stage autonomous ML pipeline.

The system separates the research process into four roles:

Stage	Responsibility
Data Processor	Reads the task, dataset structure, and available recommendation signals
Data Cleaner	Checks feature quality, removes risky signals, and guards against leakage
Training Model	Trains recommender candidates and produces predictions
Continual Learner	Reviews metrics, accepts or rejects candidates, and chooses the next experiment

For the KuaiRand-Pure benchmark, EpsilonRec connects this framework to a real recommendation pipeline. It trains and evaluates candidate models using validation and test outputs, then records results in reproducible files such as benchmark reports, metric summaries, submission CSVs, and experiment logs.

We tested multiple candidate directions, including:

factorization-machine baselines
multi-behavior features
time-decay signals
pairwise ranking approaches
retrieval-then-ranking ideas
exposure-aware diagnostics
multitask recommender variants

Each candidate was evaluated through the same process, so the system could compare ideas fairly instead of relying on intuition.

Challenges We Faced

One major challenge was keeping the project honest.

It would be easy to make an AI dashboard that simply claims to improve recommendations, but benchmark systems need measurable proof. We therefore separated the planning interface from the actual KuaiRand-Pure evaluation pipeline. The dashboard can explain and organize experiments, but the benchmark metrics decide whether a model is actually better.

Another challenge was that many promising ideas did not improve validation performance. Some advanced candidates, such as pairwise ranking and multitask variants, sounded strong conceptually but underperformed in measured results. Instead of hiding this, EpsilonRec records those failures and rejects the candidates.

This became one of the most important parts of the project: the agent is not only autonomous when it succeeds, but also when it decides not to trust a bad experiment.

What We Learned

We learned that autonomous ML is not just about generating code or trying random models. A useful autonomous research agent needs structure, constraints, and feedback.

The most important lessons were:

Recommender-system improvement requires repeated experimentation.
Validation metrics must guide decisions, not model complexity.
Failed experiments are valuable if they are logged and used to guide the next step.
A strong baseline can outperform more complicated methods.
Autonomy is safer when the agent chooses from controlled experiment options instead of making unlimited arbitrary changes.

EpsilonRec helped us understand that the real value of AI in ML research may not be replacing the model, but accelerating the research loop around the model.

Accomplishments

We are proud that EpsilonRec is more than a static recommender submission.

It demonstrates:

an autonomous experiment loop
real KuaiRand-Pure benchmark integration
measured GAUC and nDCG@5 evaluation
candidate comparison and rejection logic
reproducible benchmark logs
generated validation and test submission files
a dashboard that explains the research process clearly

Most importantly, EpsilonRec shows how an ML system can improve through evidence rather than guesswork.

What’s Next

Next, we want to expand EpsilonRec into a stronger long-running research agent.

Future improvements include:

running longer autonomous experiment cycles
testing up to 50 iterations over a fixed time budget
adding stronger feature engineering for user-video behavior
improving pairwise ranking and calibration
adding cached feature tables for scalability
supporting retrieval-then-ranking for larger candidate sets
improving the continual learner’s experiment selection strategy

Our goal is for EpsilonRec to become a zero-human-intervention recommender research agent: one that can keep proposing, training, evaluating, revising, and learning from its own benchmark results.

Final Summary

EpsilonRec is not one recommender model. It is the researcher behind the recommender.

By turning KuaiRand-Pure recommendation development into an autonomous loop of hypothesis, training, evaluation, and revision, EpsilonRec makes recommender research more scalable, measurable, and reproducible.