Inspiration
Slot machines look random, but a simulator is never truly random: it has rules. We wanted to see how close we could get to the simulator's true payout odds using only the logs. The scoring rewards calibrated probabilities, not lucky guesses.
What it does
For every query (an episode of 8 machines and a payout history up to a cutoff tick), it outputs calibrated probabilities for the next payout: 0×, 1×, 2×, 5× or 20×. The same pipeline refits itself automatically on new data, which is what the adaptation round needed.
How we built it
- Exploratory analysis → recovered generator structure: a machine's win chance depends on wins at specific lag pairs (an XNOR-style rule), and payout size follows a regime set by earlier big payouts.
- Bayesian cell model: Beta-pooled win rates per (machine type × lag-pattern cell), mixed over episode types (stable, drift, shared change). A generic fallback model is blended in for robustness.
- Hidden shared state: a two-state HMM ("hot/cold" weather) that moves all machines of an episode together, fitted by EM. Wins of other machines a few ticks earlier enter as a coupling term.
- Short histories: missing win and size categories are integrated out instead of ignored.
- Honest validation: parameters are fitted on train only and checked once on validation. Episode-level cross-validation guards calibration, and pseudo-short-history tests are scored against realised outcomes.
- Adaptation round: one script re-detects the lag pairs, size lags and coupling lag on the new kit, refits everything, and adopts a component only if validation improves. It was done within the 90-minute window.
- Inference is pure Python standard library. The code ZIP reproduces our CSV exactly.
Challenges we ran into
- At one point we wrongly concluded we had hit the noise floor. A second look found two more mechanisms (the size regime shifts the win odds, plus a shared episode state), worth about +0.8 points.
- Avoiding overfitting to a small public validation set: we used episode-clustered standard errors and train-only fits.
- Keeping runtime and reproducibility within the 30-minute, 2-CPU rerun limit.
Accomplishments that we're proud of
- Main round: 59.12 / 60 on held-out validation (the pooled starter scores 40 of 80, i.e. 30 on this scale).
- Adaptation: 19.74 / 20 on the new kit's validation, with the pipeline adapting automatically to the changed rules.
- Reproducible, dependency-free inference.
What we learned
Proper scoring rules reward modelling the data-generating process itself, not just predicting outcomes. Being skeptical of "we're done" pays off, and so does an evaluation you can trust.
What's next for HouseEdge-3Lsahur
Short-history episodes are where most of the remaining error is. Next steps would be a better joint posterior over unseen ticks, and a fully online model that detects rule changes as they happen.
