Gooners

Project Overview & Solution Addressing the Problem

The KuaiRand-Pure benchmark evaluates within-user recommendation ranking over logged short-video impressions using the binary long_view engagement label. Performance is scored per-user with Group AUC (GAUC) and Normalized Discounted Cumulative Gain at rank 5 (nDCG@5), combined into a single primary metric:

$$\text{Primary} = \frac{\text{GAUC} + \text{nDCG@5}}{2}$$

To address this, we built an end-to-end framework plus an Autonomous ML Research Agent that iteratively develops, evaluates, and tunes recommendation pipelines with minimal human intervention:

Baseline Benchmarking — Establishes reference performance on Random (Validation Primary: 0.4834), Item Popularity (0.5807), and the official single-CPU Factorization Machine baseline (0.6016).
Pairwise Ranking Optimization — Shifts training from pointwise binary cross-entropy loss (baseline.py) to pairwise Bayesian Personalized Ranking loss (bpr.py), directly optimizing the within-user score differential ($\text{score}{\text{pos}} - \text{score}{\text{neg}}$) that GAUC/nDCG actually reward.
Feature Space Engineering — Extends categorical inputs from a basic 5-field set (user_id, video_id, author_id, tab, dur_bucket) to a 9-field item-side representation (adding music_id, video_type, upload_type) and a 13-field full representation (adding user demographics like register_days_range, user_active_degree).
Autonomous Agent Loop — agent.py runs a closed-loop search over learning rates: it launches each experiment as a subprocess, parses validation GAUC/nDCG/primary from stdout, logs every run to experiment_log.json, and proposes the next learning rate from the best result so far — up to 8 experiments, no manual retuning in between.