OmniRec: Autonomous RecSys ML Research Agent

A universal autonomous RecSys research agent with adversarial peer-review & 4-pillar taxonomy that iteratively invents, optimizes, and converges PyTorch models.

Comment

💡 Inspiration

Recommender Systems (RecSys) are among the most complex domains in machine learning, requiring continuous exploration across sparse feature representations, multi-task objectives, and listwise ranking formulations. Traditional AutoML often resorts to brute-force hyperparameter sweeps or stalls in localized optima. We built OmniRec: a universal, first-principles autonomous ML research agent inspired by human peer-review and academic discovery. It formulates deep hypotheses, subjects them to adversarial scrutiny, writes clean PyTorch CUDA code, and systematically navigates multi-paradigm research spaces until formal convergence without human intervention.

🚀 What It Does

OmniRec operates as an autonomous research loop governed by four foundational components:

  1. 4-Pillar Universal RecSys Taxonomy: Instead of random trial-and-error, the agent systematically rotates across four universal recommendation pillars:

    • Pillar 1: High-Order Explicit & Attention Crosses (DCN-v2, CrossNet, LayerNorm stabilization).
    • Pillar 2: Multi-Task & Multi-Feedback Learning (Progressive Layered Extraction / PLE, Auxiliary engagement & retention heads).
    • Pillar 3: Feature Representation & Statistical Priors (Historical user/item Empirical Bayes CTR priors).
    • Pillar 4: Ranking Loss Optimization & Listwise Formulations (ApproxNDCG, LambdaLoss, Softmax CE).

  2. Research Critic (Adversarial Peer Reviewer):

    • Vetoes trivial micro-tweaks (e.g., minor learning rate shifts) when a paradigm plateaus.
    • Evaluates mathematical alignment against the target top-K ranking metric: $$\text{Primary Score} = \frac{\text{NDCG@10} + \text{Recall@50}}{2}$$
    • Audits code safety to prevent label leakage and tensor shape mismatches.

  3. In-GPU VRAM Zero-Copy Engine:

    • Pre-loads large-scale categorical interaction tensors directly into CUDA memory, eliminating CPU-to-GPU PCIe bottlenecks and completing 40-epoch training passes in ~1.2 seconds.

  4. Multi-Pillar Global Convergence Protocol:

    • Enforces the mathematical Rule of $\epsilon$: the agent only declares convergence if improvements satisfy $\Delta \le \epsilon$ across consecutive iterations after completing valid explorations in all four research pillars.

🛠️ How We Built It

  • Core Language & Frameworks: Python 3.10+, PyTorch 2.0+ (CUDA accelerated), NumPy, Pandas, Scikit-Learn.
  • LLM Reasoning & Orchestration: Gemini API with exponential backoff and rate-limit aware retry loops.
  • Safety Guardrails: Static AST linters and automatic Git rollback controllers to instantly revert code regressions.
  • Architecture Innovations:
    • Progressive Layered Extraction (PLE) separating shared and task-specific expert sub-networks across click, like, and long_view signals.
    • Layer Normalization integrated directly inside high-order cross-layers to stabilize gradient flow.
    • Empirical Bayes Item CTR priors combined with sequential user interaction embeddings.

🏆 Results & Accomplishments

  • Organizer Baseline (Factorization Machine): Primary Score = 0.5946 (Test) / 0.5641 (Val).
  • OmniRec Winning Model: Primary Score = 0.6175 (GAUC = 0.6690, nDCG@10 = 0.5660).
  • Performance Gain: +0.0229 over baseline (+3.85% relative) with a +7.16% relative gain in nDCG@10.
  • Efficiency: 80% prompt token reduction via dense history milestone summaries and 1.2s CUDA epoch execution.

🧗 Challenges We Overcame

  1. CPU DataLoader Bottlenecks: Python dictionary loops across 125,000 validation items caused 10s per-epoch delays. We engineered a direct in-GPU tensor slicing engine, cutting runtimes by 10x.
  2. Listwise Ragged Tensor Bugs: Differentiable ranking surrogates often fail on variable-length candidate sets. The Critic enforced fixed-size padded tensor contracts that eliminated all runtime crashes.
  3. Preventing Premature Stagnation: Replaced naive local convergence with a paradigm plateau detector that forces the agent to pivot to Multi-Task Learning whenever interaction layers saturate.

📚 What We Learned

Autonomous ML agents succeed when domain constraints (RecSys first-principles) are baked into the agent's taxonomy and peer-review prompts. Combining adversarial scrutiny with fast, deterministic execution harnesses allows LLMs to act as true empirical scientists rather than stochastic code generators.

Built With

Share this project:

Updates

Submission history