Inspiration

The brief opened with a decision that sounded obvious: management is about to replace the ten machines that earned the least. Sort by house win, take the bottom ten, swap them out. We wanted to know whether that list really measured bad machines, or whether it was mostly measuring where they sat, how long they had been on the floor, and luck.

Casinos are a hard place to tell signal from noise. A single slot machine's result over a few months can swing tens of thousands of dollars on randomness alone. A dashboard that ranks machines by raw win treats that noise as performance. We wanted one that pulls the two apart, and that a floor manager could still read in two minutes.

What it does

Luck, Location, or a Leak? is a single-file, offline, interactive dashboard covering six months (H1 2025) at The Gilded Ace, a fictional 300-machine casino. Its answer: don't replace the bottom 10.

  • Machines: 9 of the 10 are in Back Forty, the zone farthest from any entrance. 2 were already retired and 1 was installed in May. Their \$541k gap to typical machines breaks down as 36% game design, 30% zone, 20% exposure (days on the floor) and 12% luck. The cabinets themselves beat their same-game peers by 4%.
  • The leak: one machine, M-1158, hits on 38.6% of spins against a configured 31.5% (\(z = 124\)) and paid \$36k above expectation. We flag it for inspection, not as an accusation.
  • Demand: a walkway closure (Mar 3 to Apr 7) cut Back Forty by 31%, but play moved to other zones, so the floor as a whole lost almost nothing. June softened because it had fewer weekend days, 15 days above 40 °C, and lower tourism.
  • Downtime: availability was 99.62%. 96% of interrupted players kept playing on another machine. An April–May slot-tech shortage doubled repair waits and was fixed by June.
  • Players: card-holders wagered \$163.8M, got \$148.2M back and lost \$15.6M, almost exactly the expected \$15.5M. 16.1% finished ahead, which is what luck predicts. Every player who lost \$2,500+ is a local, so we propose opt-in loss limits only, never targeting.
  • Recommendation: spend \$24,000 of the \$100,000 budget on three actions: inspect M-1158 and two Comet Cash machines, move three Back Forty machines into empty prime slots, and run one tournament weekend. Keep \$76,000. Each action has a low/mid/high effect estimate and a way to measure it.

Judges can change the analysis period, filter by zone, bank or guest group, inspect how the model was validated, and click Evidence → on any recommendation to jump to the chart behind it. Every chart has a table view, keyboard access, and light and dark themes.

How we built it

Data and analysis. We profiled the parquet tables with DuckDB and wrote one Python script per question (q1_machines.py to q5_comps.py, plus model_forecast.py). Each script writes a JSON file. A build script turns those files into the dashboard and the Claims & Evidence appendix, so every number in the write-up matches the number on the screen.

Separating luck from performance. The key idea was to compare each machine's actual house win \(W\) with its theoretical win \(T\) (the luck-free expected win from the game's configured RTP), and to scale the difference by the payout variance the machine actually saw:

$$ z = \frac{W - T}{\sqrt{\sum_{d} \left(\sum_{i} b_{i,d}^{2}\right)\sigma_{\text{game}}^{2}}} $$

Here \(\sum b^2\) is the daily sum of squared bets and \(\sigma^2_{\text{game}}\) is the game's payout-multiplier variance. If every machine is honest, these z-scores should look standard normal. Across 310 machines they did (SD 1.07, KS \(p = 0.32\)). Only one machine survived Benjamini–Hochberg correction: M-1158.

Explaining the gap. We built a leave-one-out benchmark of theo per available hour for the same game at comparable slots. We then used a log-mean Divisia decomposition to split the manager's \$541k gap into game, zone, exposure, luck, closure, bank and cabinet effects.

Causal checks. For the walkway closure we used difference-in-differences with a day-block bootstrap. We also ran 78 placebo windows; they averaged −0.2%, against −31% for the real closure. The promotion and comp-price analyses used matched baselines and interrupted time series with placebo dates. Fault rates by manufacturer used a quasi-Poisson GLM.

The model. We froze a calendar regression (weekday + pre-announced events + holidays) on April 30 and scored it once on May–June against 3 baselines and 5 other candidates:

Method May–June MAE / day
Calendar regression \$45k (3.5%)
Weekday means \$69k
Constant mean \$218k

The model has a job in the recommendations. It showed that a naive event-vs-average comparison overstates a promotion's value by 70%. It also becomes the July baseline the tournament weekend will be measured against.

Dashboard. The dashboard is a self-contained HTML file. The charts are hand-built SVG in vanilla JavaScript, with no libraries and no server, so it opens by double-clicking. Every KPI recomputes from additive monthly or daily sums when the filters change. We tested it in headless Chromium in light and dark mode, at desktop and mobile sizes, including filter and evidence-link interactions.

AI assistance. We used Claude Code (Claude Opus 5.5) throughout: for exploration queries, the analysis scripts, the dashboard and first drafts of the writing. We cross-checked headline numbers with independent queries, re-ran every script end-to-end from raw data, and passed the organizers' verification scripts. Full details are in our AI-use disclosure.

Challenges we ran into

  • Luck is huge and lumpy. High-volatility games such as Jackpot Summit can swing wildly over short periods, and the normal approximation is rough there. We had to use each game's real payout variance rather than a single floor-wide one, and we cross-checked the luck test with a separate hit-rate test. M-1158 failed both. Everything else passed both.
  • Multiple testing. With 310 machines, about 15 will look "significant" at 95% by chance alone. We saw 24 beyond \(|z| > 1.96\). Only Benjamini–Hochberg correction separated one real anomaly from the noise.
  • Reporting the result we didn't want. Our inner April validation actually picked a different model variant (one with a capacity term), and it scored worse on the holdout (\$59k MAE). We reported that honestly rather than quietly swapping in the winner after the fact.
  • Location and placement aren't randomized. Good games tend to get good slots. We leaned on the four machines that moved during the window: each took on its new location's earning rate, which is evidence that the slot, not the cabinet, drives demand.
  • Saying "keep the money." It's tempting to spend the whole budget. Pricing each action with the guide's lead-day formula showed that replacing the bottom ten loses money in July (leasing nets about −\$13k). The most defensible move was to spend less than a quarter of the budget.
  • Time. We cut two planned side analyses (an anomaly scan of the optional tables and a broader search for natural experiments) to meet the deadline.

What we learned

  • Rank by theo per available hour, not raw win. Raw win mixes four things a manager can't see apart: design, location, exposure and luck.
  • A test for luck is a test for everything else. Once you know what randomness alone should look like, real anomalies stand out sharply. A \(z\) of 124 is not a bad month.
  • Local effects aren't always global. The walkway closure was a disaster for one zone and a non-event for the casino, because players moved.
  • Validation protocol matters as much as the model. Freezing the forecast before the holdout, testing against baselines with Diebold–Mariano, and using block bootstraps made our 34% improvement claim defensible.
  • Generate the write-up from the data. Building the claims appendix from the same JSON as the dashboard removed a whole class of copy-paste errors.

What's next

Run the July tournament weekend against the frozen forecast, inspect M-1158 and the Comet Cash fleet, and pilot a comp-price reversal in August with a control group, which the May 1 floor-wide change never had.

Built With

Share this project:

Updates

Submission history