HERE'S THE GOOGLE DRIVE LINK FOR THE ZIPPED FILES FOR THE ADAPTATION ROUND - https://drive.google.com/file/d/1n-UyXF92UmVu049wnyYSgkgRiUhaHA0a/view?usp=sharing
Inspiration
House Edge asked us to forecast the odds of a slot machine's next payout (0×, 1×, 2×, 5× or 20×) from gameplay logs. We were scored on how close our probabilities came to the simulator's true odds, not on the spins that happened. The starter models count payouts, and none of them got far past the frozen reference (about 30 of 60 points). The rules hinted at more: outcomes "interact at lags up to eight ticks", and machines of the same type "can respond differently or oppositely." We suspected the simulator had real structure, and that if we found it we could beat the baseline by far more than tuning would.
How we built it
1. Infrastructure first. We built one example table: every observed tick becomes a "predict tick $t$ from ticks before $t$" example, which gave us about 280k training rows instead of 768 labels. We also wrote an evaluator that reproduces the competition score exactly:
$$R = \frac{1}{4}\sum_{c}\frac{E_c}{B_c}, \qquad \text{points} = \frac{60}{1+R}$$
Here $E_c$ is our mean squared probability error in condition $c$ (averaged within episodes first) and $B_c$ is the reference's.
2. Experiments, each with a null control. We ran nine experiments on lags, encodings, neighbors, non-stationarity and short histories. Each used leave-one-out count models and a control that shuffles the feature within each machine. Single lags looked weak. Pairs of lags were dramatic.
3. Reverse-engineering with the validation answers. Validation includes the true probabilities, so we could line them up against each machine's state at the cutoff. The simulator turned out to be simple and exact:
- A matching-pair rule. Each machine has one lag pair $(i, j)$, in practice $(1,3)$, $(2,5)$ or $(4,7)$. The rule state is $$s_t = \mathbb{1}\left[w_{t-i} = w_{t-j}\right]$$ where $w_t$ is win/loss. One side of the rule is favorable: about 95% vs 15% win chance for steady machines, about 59% vs 1.4% for jackpot machines. Which side is favorable is a coin flip per machine.
- Signal adds about $+0.35$ to the log-odds of a win per unit.
- The conditions describe how the rule changes over time. In drift, each machine's rule flips direction once, abruptly, at its own tick. In shared change, every machine in the episode flips on the same tick. Stable episodes never flip.
- Payout sizes: given a win, the $2\times:5\times:20\times$ split is exactly fixed per machine type. Only the 1× share varies.
- Hidden state: an episode-wide offset on the log-odds of winning that slowly wanders over time.
4. A model that mirrors the generator. For each machine we enumerate every hypothesis $h = (\text{pair}, \text{direction}, \text{flip tick or none})$ and score it with Beta–Bernoulli marginal likelihoods:
$$p(h \mid \text{history}) \propto p(h)\,\frac{B(\alpha_f + k_f,\ \beta_f + n_f - k_f)}{B(\alpha_f, \beta_f)}\,\frac{B(\alpha_u + k_u,\ \beta_u + n_u - k_u)}{B(\alpha_u, \beta_u)}$$
Here $k$ and $n$ count wins and spins in the favorable and unfavorable cells. The forecast is the posterior-weighted win chance at the cutoff, adjusted by shrunk episode-level and machine-level log-odds offsets, then combined with the type's fixed size mix. Every prior (pair frequencies, flip timing, state rates by type and signal) is fitted on the train split only.
5. Benchmarking against machine learning. We tested gradient boosting and an MLP head-to-head, including versions given features built from the rules we had discovered. The best scored 53.6 points, below the Bayesian model, and blending or calibration added nothing measurable. The model is also exact, deterministic and fast, which matters for the organizers' rerun.
Result: 55.61 / 60 on validation ($R = 0.079$), against 30.00 for the reference. The submission is pure-Python standard library, runs in about 8 seconds, and a fresh-folder rerun reproduces it byte for byte.
What we learned
- Use the strongest signal you have. True probabilities at the cutoff turned noisy outcome data into an exact read on the simulator. Two numbers per forecast, a fixed size split per type, and one flip per machine were all invisible in summary statistics.
- Every test needs a null. Several early "signals" vanished once we compared them against shuffled controls.
- Inference beats function approximation here. The hard part is identifying a hidden rule per machine from about 250 spins, with only 768 training machines. A model that does that inference explicitly beat learners trying to approximate it.
Challenges
- Misleading averages. Pooled statistics hid the rule: machines split 50/50 between "match favorable" and "mismatch favorable", which cancels out in any pooled correlation. Per-machine analysis was essential.
- Short histories. These machines have only 64 observed ticks, and only about 32 of them have all eight lags. Rule identification there relies heavily on priors, and it's still our weakest condition.
- Scoring with noisy outcomes. The irreducible noise in observed outcomes is about twice the reference's true error, which squeezes outcome-based score ratios toward 1. We had to confirm every ratio against the exact validation targets.
- Reproducibility constraints. The rerun environment is the Python standard library on 2 CPUs, with a 1e-8 tolerance. We ported the numpy prototype to pure Python using prefix sums and
math.lgamma, and matched it to within $10^{-14}$.
