Autonomous ML Research Agent for RecSys workflow

Submitted for TikTok Techjam 2026, track #2.

This project is an autonomous ML research agent for the TikTok TechJam 2026 KuaiRand-Pure recommendation task. It preserves a gated research workflow: preflight, official FM baseline reproduction, evidence-driven planning, experiment execution, submission validation, official scoring, promotion or rejection, convergence, and finalization. This ensures model improvements are selected reproducibly rather than by manual trial and error.

The agent trains only on interactions dated 2022-04-08 through 2022-04-21 and uses validation labels from 2022-04-22 through 2022-04-28 for evaluation and model selection. Test rows from 2022-04-29 through 2022-05-08 may be used only to produce structurally aligned predictions; test labels are not loaded into training, feature fitting, planning, tuning, promotion, convergence, or reporting.

Every candidate produces row_id,user_id,video_id,score, passes row-order, identity, row-count, and finite-score validation, and is then evaluated with the organizer implementation of GAUC and nDCG@5. The official selection value is primary = (GAUC + nDCG@5) / 2. The baseline must reproduce the organizer validation reference before experiments can start. The agent promotes only a strictly better validation primary, tracks the validation-best checkpoint independently of the last experiment, and stops through formal convergence or a distinct hard-budget/catalog condition.

The planner receives an evidence pack containing baseline and incumbent metrics, prior hypotheses, code/config change summaries, outcomes, failures, recovery events, and remaining budgets. It never receives test labels or hidden-test scores. LLM-backed planning can choose reviewed experiments and refine only bounded parameter values declared in configuration; executable commands and benchmark policy remain under orchestrator control.

Development tools

Python 3.11+ command-line development and testing
Git for version control and diff/audit evidence
PowerShell and Ubuntu/WSL shell scripts for setup, execution, and reproducible commands
Codex for assisted repository analysis and implementation
The Understand Anything code-graph tooling for architecture tracing during development

The repository does not require Colab, Jupyter notebooks, or a particular editor. It can be developed in VS Code or any editor with Python support.

APIs

NUS School of Computing OpenAI-compatible Chat Completions API for the default optional LLM planner (qwen3.8:27b)
OpenAI Responses API as an optional alternate planner provider
Google Gemini GenerateContent API as an optional alternate planner provider

API use is optional: the deterministic catalog planner runs locally. API keys are read from environment variables or the ignored .env file and are not copied into run artifacts.

Libraries and frameworks

Python standard library for orchestration, configuration, subprocess control, CSV/JSON I/O, HTTP integration, reporting, and tests
NumPy for the organizer-style Factorization Machine baseline and efficient feature encoding
unittest for contract, metric, convergence, reliability, leakage, submission, and end-to-end tests

The core workflow intentionally does not require pandas, scikit-learn, PyTorch, TensorFlow, or Hugging Face Transformers. Those libraries may be introduced by future experiments if pinned and justified, but they are not required by the current baseline/control plane.

Datasets and assets

KuaiRand-Pure interaction logs:
- training: 2022-04-08 to 2022-04-21
- validation: 2022-04-22 to 2022-04-28
- test structure only: 2022-04-29 to 2022-05-08
KuaiRand-Pure user and video feature tables supplied with the public dataset
Organizer-provided baseline.py, evaluate.py, and submit.py behavior for the FM reference, GAUC/nDCG@5 scoring, and submission alignment checks
Organizer validation baseline references: GAUC 0.6674, nDCG@5 0.5357, primary 0.6016
Repository recommender-system knowledge notes used as contextual planning evidence, subordinate to the official benchmark contract