About the project
Inspiration
Every slot machine looks random to the person pulling the lever. The House Edge challenge gave us gameplay logs from simulated machines and asked us to predict the probability of each payout on the next spin: $0\times, 1\times, 2\times, 5\times$ or $20\times$. We were scored against the simulator's true probabilities, so lucky guesses on sampled jackpots couldn't help.
That made it a reverse-engineering problem. If a program generated these machines, its rules leave fingerprints in the data. We wanted to find those rules, not fit a black box on top of them.
What we built
We built a joint Bayesian model that inverts the simulator. It is written in pure Python with no dependencies, and it predicts all 1,536 test queries in about a minute.
We found that each machine's chance of winning follows a logistic rule:
$$ \operatorname{logit} P(\text{win}t) = b{\text{type}} + K\, s\, m(t)\, X_t + \beta\, \text{signal}_t + \gamma\, Z_t + \delta\, H_t + \lambda\,(C_t - c_0) $$
$X_t = \text{XNOR}(\text{win}{t-i}, \text{win}{t-j})$ is a signed interaction of two past wins. Each machine uses one hidden lag pair $(i,j) \in (1,3), (2,5), (4,7)$ and its own sign $s = \pm 1$.
$Z_t$ marks whether a recent payout was big ($\ge 2\times$). It also chooses between a "high" and a "low" payout-size profile.
$H_t \in 0,1$ is a hidden hot/cold state shared by the whole episode. It follows a two-state Markov chain with switch rate $\rho = 0.01$, and we infer it with forward–backward filtering.
$C_t$ counts how many other machines in the episode won 3 ticks earlier. This cross-machine effect added 38 nats of training log-likelihood; every other lag added less than 1.
$m(t)$ is $+1$ before a regime flip and $-1$ after it. Drift flips ramp down gradually over about 20 ticks, while shared changes flip all at once.
We don't know which rule a given machine uses, so the model averages over every plausible combination of lag pair, size lag and sign. Each episode is also weighed under three explanations: no change, per-machine drift and a shared flip tick. The prediction is the posterior-weighted mixture:
$$ \hat{p}(y) = \sum_{h,\,c,\,\tau} P(h, c, \tau \mid \text{history})\; p(y \mid h, c, \tau) $$
Here $h$ is the rule hypothesis, $c$ is the condition and $\tau$ is the flip time. Short-history episodes are detected from how many ticks were observed, never from labels, and they get a strong "no change" prior.
How we built it
Exploration. We wrote small probing scripts to check which lagged outcomes predicted wins. Clean, sign-flipping XNOR patterns jumped out.
Prototype. A per-machine hypothesis filter took us from the 30-point reference to about 54 points.
Joint model. Adding the shared hot/cold state and averaging over the three condition explanations reached about 58.5 points.
Train-only fitting. fit_train.py discovers the lag structure and fits every parameter by hard EM with a logistic M-step, using only training histories. Validation answers were used only to choose a few settings and to score the model, and the README says exactly what each data split was used for.
Refinement. The cross-machine term, gradual drift and the short-history prior took us to 59.03 / 60 on validation.
Reproducibility. One command, python predict.py --data /data --out /output/submission.csv, rebuilds the exact CSV with no randomness. Refitting from train reproduces params.json byte for byte. We tested this in a read-only, offline setup like the organizers' rerun.
Scoring
For each condition $c$, the error is the episode-averaged squared probability error $E_c$, divided by the error $B_c$ of a frozen reference model:
$$ R = \frac{1}{4}\sum_{c} \frac{E_c}{B_c}, \qquad \text{points} = \frac{60}{1+R} $$
Our validation loss ratios were 0.007 (stable), 0.039 (short history), 0.012 (drift) and 0.007 (shared change). That means our error was roughly 1–4% of the reference model's.
Challenges
Hidden state everywhere. The rule, the sign, the flip time and the hot/cold state are all unobserved, and they explain each other away. Getting full Bayesian averaging to run fast enough meant precomputing prefix log-likelihoods for every hypothesis and flip time.
Validation discipline. An earlier version fitted parameters directly on validation answers. We rebuilt the pipeline so every parameter comes from training data alone. Then we checked that scores didn't drop.
Short histories. With few observed ticks, a drift and noise look the same. The fix was a prior based on how much history was observed, not more model complexity.
Exact reproducibility. The rerun allows only a $10^{-8}$ difference. Our CSV matched byte for byte, and other Python versions differed by about $5\times10^{-15}$.
What we learned
When data comes from a program, finding its structure beats adding model capacity. Our biggest gains came from discovering a mechanism, not from tuning.
Bayesian model averaging is a strong default when several stories fit the data. It handles uncertainty about what is happening instead of betting on one guess.
Being strict about how each data split is used is what makes a validation score trustworthy.
A one-command, dependency-free reproduction is worth building early, because it removes the last-minute risk.
Submission files
Team: beta-dancers
Round 1 — Main (submitted, unchanged)
submission.csv — SHA-256 608859f7bb53c3f0388f4a066f592a1233105317cf3823bd0f4d103f2ef64669
entry.zip — SHA-256 a16f31f174b10df8e147bbfe64e6d3d940ea3396905754d4e305f4a1638f63ae
Code: GitHub release r1
Validation score (official checker): 59.031 / 60
Reproduce: python predict.py --data /data --out /output/submission.csv (Python standard library only, no requirements.txt needed)
Round 2 — Adaptation
submission.csv — SHA-256 b82a49fd5f8aafe660e4202513a17f8c860474cf6e5c03545e75a2e7e3719033
entry.zip — SHA-256 eec71d682725ed56850c57c2f66aab7df523edbab99958a3edc194f9af2b1190
Code: GitHub release r2 (folder adaptation/)
What changed: the hidden win rules moved. Our rule miner rediscovered the new interacting lag pairs from adaptation train alone: $(1,3),(2,5),(4,7) \to (3,8),(1,4),(2,6)$, with size lags now $3, 1, 2$. The cross-machine lag stayed at 3.
How we adapted: same code, no hand edits. fit_train.py rediscovered the structure and refitted params.json on adaptation train in about 2.5 minutes.
Why it mattered: the unchanged main model scored only 7.69 / 20 on adaptation validation, worse than the reference. After the refit it scored 19.79 / 20 on the kit scale, or $\approx 39.6 / 40$ on the 60/40 split.
We also tested 16 setting changes on validation. All of them landed within $\pm 0.03$ points, so we kept the main-round settings rather than overfit.
Reproduce: python predict.py --data /data --out /output/submission.csv (Python standard library only)
Projected total on validation (60/40): $59.03 + 39.58 \approx 98.6 / 100$.
BUILT WITH (Devpost tags, enter one by one): python, bayesian-inference, hidden-markov-models, expectation-maximization, logistic-regression, forward-backward-algorithm, probabilistic-modeling, claude-code, git, github
