Inspiration
The contest isn't "who wins the next spin." For each simulated slot machine we had to say how likely each payout is on its next spin (nothing, 1×, 2×, 5× or 20×), and the score compares our guess with the simulator's true probabilities. That's harder than it sounds. Machines drift partway through, some have very little history, and big wins are so rare that it's easy to fit noise and feel clever.
What it does
For every machine at its cutoff, we output five probabilities. Three kinds of model each make a forecast, and we blend them:
- M3 is a small Bayesian model fitted separately to each machine. It considers rules like "the machine wins when the spin two ticks ago and the one before it differ," weighs them against each machine's history, and allows for a flip point where the rule reversed (that is what drift looks like in the data).
- 12 GRU networks read each machine's history one tick at a time and predict the next tick. They are trained to predict the win/no-win part and the payout size separately, with the size term weighted twice as heavily.
- 8 small transformers (128 wide, 3 layers) that look at a causal window of ticks with positional biases (ALiBi), so a tick can only attend to earlier ones.
We blend the chance of winning and the size of the win separately, because the models disagree about each in different ways.
How we built it
- Prediction runs on CPU with NumPy and SciPy in about 4 minutes, with no PyTorch needed. Training used PyTorch.
- The neural models learn from the history of every machine in the released data (only ticks before each cutoff), plus the training labels. They never see validation or test answers.
- Blend weights come from 4-fold cross-validation, where whole episodes are held out together. We never choose weights on the episodes we score.
- Every packaged entry has to reproduce its own CSV exactly from a fresh folder, offline, in under 30 minutes on 2 CPU cores.
What we measured
Scores are points out of 60 for round one (the plain frequency baseline scores 30) and out of 40 for round two (the baseline scores 20).
Round one: models, built up one idea at a time (Amir's branch, validation set, so a bit optimistic)
| Model | Points |
|---|---|
| Pooled frequency baseline | 30.00 |
| Frequency model with tuned shrinkage and recency | 31.09 |
| Lag-pair rules per machine (Bayesian) | 52.60 |
| + a flip point where the rule reverses (M3) | 55.27 |
| + one flip shared by the whole episode | 55.28 (no real gain) |
| + gradient-boosted corrector | 54.99 (worse) |
| Weighted blend of models 2–5 | 55.30 (no real gain) |
Round one: ensembles and the final blend
| Entry | Points |
|---|---|
| Even blend of M3 and 12 GRUs | 56.17 |
| Win 70% M3 / 30% GRUs, size from GRUs | 56.58 |
| + 8 transformers | 56.90 |
| Same, after fixing a signal-timing leak (clean refit) | 56.75 |
| Final entry, 4-fold cross-validated (out-of-fold) | 57.28 |
| Simpler recipe, same cross-validation | 57.11 |
Round two (adaptation), 96 validation episodes, 40-point scale
| Blend | Points |
|---|---|
| Our fixed blend | 38.19 |
| Equal thirds | 37.86 |
| Transformers alone | 37.45 |
| M3 alone | 37.01 |
| GRUs alone | 36.39 |
Every single model is beaten by the blend. Re-tuning the weights with k-fold on the new validation set gave +0.04 (interval −0.006 to +0.092), which is noise, so we kept the weights fixed.
What did not help: mixture-of-experts models (tied with an extra GRU of the same size), training 150 epochs instead of 100 (the held-out loss got worse on all three seeds), and merging more models for gains under 0.1 point.
Challenges
- Data volume, not memorization. The GRUs scored 34 points trained on 96 episodes of history and about 55 trained on 384, so the lesson was to give them every history we're allowed to use.
- Leaks. We caught several, mostly in how a late-arriving signal got attached to earlier ticks. We built a leak test for each model and fixed three before submitting.
- Honest scoring. Gains under about 0.1 point are noise, so every comparison used paired bootstraps over whole episodes, and we wrote the decision rule before running each test.
- Round two. The organizers' scoring scale and checker didn't agree, so we used the adaptation kit's own checker.
What we learned
Episodes, not ticks, are the real sample size. A simple per-machine Bayesian model gets surprisingly close to the neural ones, and the neural ones add most when blended with it. A negative result with a rule written before the run is worth more than a lucky score.
AI tools
We used Claude Code and Codex as coding assistants. The forecasting models are small networks and Bayesian models trained from scratch, not generative AI.
