Inspiration
Our Dashboard Royale entry answers a casino GM's question: should the ten lowest-earning slot machines be replaced? The analysis worked, but it ran as hand-ordered Python scripts on a laptop. Checksums were checked once, nothing enforced data rules, there was no quality history, and nobody but the analyst could ask the data a question. We wanted to see what a real lakehouse would add: the same answers, with governance, automation, experiment tracking and plain-English access.
What it does
We rebuilt the whole machine-replacement (Q1) pipeline and the demand forecast inside Databricks Free Edition:
- Bronze: the official 31-file Parquet release lands in a Unity Catalog volume. 31/31 SHA-256 checksums and 31/31 row counts match the release manifest, for 3,885,259 rows in one Delta table per file with lineage columns.
- Silver:
silver_floor_dayenforces 5 Delta CHECK constraints (availability ≤ schedule, occupancy ≤ availability, non-negative money, win = coin-in − payout, on-floor only).silver_machine_hourre-derives the 6 AM–6 AM gaming day from 1,321,908 UTC hourly meters with 0 mismatches, including the one 23-hour DST day. - Gold: the machine scorecard, the GM's ten, the five-part shortfall decomposition, peer ranking, zone factors and property days, all in SQL.
- Quality gate: 12 expectations per run are appended to
dq_resultsas history, and anassert_truegate fails the Job if any breaks. - Job: one 7-task DAG, bronze → silver → gold → quality → {MLflow forecast, bootstrap intervals → AI explanations}. A full run succeeds in 231.9 s. The daily 06:00 PT schedule (gaming-day close) is set up but paused.
- MLflow: every forecast candidate is a nested run. The champion reproduces our $68,698.55 holdout MAE exactly.
- Grounded AI explanations:
ai_querywith Databricks-hosted Llama 3.3 70B explains each of the GM's ten machines from a deterministic SQL diagnosis. Every cited number is checked against its evidence: 40 citations, 0 ungrounded. - Genie + AI/BI dashboard: the GM can ask questions in plain English (8/8 benchmark answers exact) or open a 12-widget dashboard that reads only gold tables.
How we built it
- Python scripts (
requests,pandas) using the Databricks REST APIs: Files API upload with checksum verification, then deployment of notebooks, the Job, the AI/BI dashboard and the Genie space. - SQL notebooks run on the Serverless Starter Warehouse; Python notebooks (forecast, bootstrap intervals) run on serverless compute.
- Parity harness:
parity.pyruns our original, unmodified Python (metrics.py,q1_machines.py,q1_decomposition.py) on the same Parquet and diffs every column against the gold tables:- Same GM bottom ten in the same order.
- Scorecard (12 metrics × 310 machines): max difference 0.0.
- Decomposition: max difference 1.5e-10 USD, main cause agreeing for 310/310 machines.
- Adjusted rank identical for 310/310, and the same single Bonferroni payout flag (M-1158).
- Bootstrap intervals (2,000 draws, seed 2026): max difference 4.4e-16.
- Diagnosis labels: 310/310 agree.
- AI safety net: each explanation must cite values as
[field=value]. A checker compares every citation with the evidence string, and failures get one automatic repair pass. In a test with 2 planted wrong numbers, the rewrite cited 4/4 values correctly from the evidence.
Challenges we ran into
- The gaming day. Casino days run 6 AM to 6 AM local time, across a DST change. We re-derived them from UTC hour stamps instead of trusting the prepared view, and reconciled hourly meters to daily totals for 56,110/56,110 machine-days.
- Exact parity, not "close enough." Reproducing pandas ratio-of-sums logic and a seeded weekly block bootstrap in SQL and notebooks took several passes. The 1-cent difference in the headline gap ($541,425.31 vs $541,425.30) comes from the original code summing values it had already rounded.
- LLMs make up numbers. We solved that with structured citations, automatic verification and a repair pass. The machine's diagnosis always comes from a deterministic SQL rule; the LLM only explains it.
- Honesty about speed: on 55K machine-days, the laptop is faster (full Q1 analysis 3.0 s locally vs 17.7 s for the gold layer on serverless SQL). The gain is governance and scale-readiness, not speed.
Accomplishments that we're proud of
- A lakehouse whose numbers provably match the original analysis, checked by an automated harness, not by eye.
- 0 ungrounded citations across 40 AI-cited numbers, plus a repair pass we tested with planted errors.
- MLflow surfaced a better model. A weekday-median challenger, also selected on the Jan–Feb → Mar–Apr validation window, cut holdout MAE to $54,885.20 (20.1% lower) with the same coverage. Because it was outside the original frozen choice set, we report it as a recommendation for the next iteration, not as a result.
- A one-command reproduction path, with every claim backed by an evidence JSON in
outputs/databricks/.
What we learned
Databricks' value on small data isn't speed. It's that the rules become part of the platform: constraints that reject bad rows, a quality history instead of a one-off script, lineage from file to gold table, scheduled runs, tracked experiments, and AI answers checked against data. We also learned that natural-language access (Genie) only works when the gold tables and business rules are clean. The governance work is what makes the AI layer trustworthy.
What's next for Dashboard Royale on Databricks
- Port Q2–Q6 (demand, downtime, guests, comps, actions), which still run in Python.
- Switch on the daily 06:00 PT schedule and land meters incrementally (Auto Loader) instead of uploading files.
- Grow the Genie benchmark beyond our 8 single-number questions, with questions written by people who didn't build the space.
- Promote the weekday-median challenger after a fresh, untouched holdout.
