About the project
Inspiration
When a team renames or changes a data column, the people who depend on it usually find out too late. Their reports crash, or worse, keep running and quietly show wrong numbers. We wanted to answer three questions before a change ships: who does it really break, what do those teams need, and how do we keep everyone to the deal?
What it does
Contract Court takes a proposed change to a table (rename, drop, retype or rescale a column) and does three things:
- Measures which downstream reports would crash or silently change.
- Negotiates: each affected report becomes an agent with its own needs, such as notice and how long it needs the old column. The owners and the teams bargain until everyone signs, or the agents hand the problem to a person when no deal is possible.
- Enforces: the deal becomes a versioned contract, and a migration that ignores it is blocked.
How we built it
- Databricks measures the blast radius. We copied a sample of every table in
samples.wanderbricks(16 tables), then generated 33 downstream reports from the schema, including reports built on other reports. For every column we applied each kind of change to a sandbox view and re-ran every report's own SQL on serverless Spark, 763 tests in all. A report is affected if it errors or returns different numbers. - Unity Catalog and Delta hold the tables, views and lineage. A Volume holds the exported results, and the notebooks log runs to MLflow.
- A Databricks model negotiates. For six high-impact changes, Llama 3.3 70B (through
ai_query) writes each team's lines and proposes the owners' offers. Fixed rules referee every deal, so invalid model output falls back to a rule-based offer. - A Streamlit app shows the story: what you have, what will happen, and the answer.
Each team's default needs come from two measurable facts, its refresh cycle $c$ (in days) and the number of report layers $d$ built on it:
$$\text{notice} = \min\bigl(30,\ \max(1, \lceil 2c \rceil) + 7d\bigr), \qquad \text{overlap} = \min\bigl(60,\ \max(1, \lceil 3c \rceil) + 7d\bigr)$$
Any team's needs can be edited in the app.
Challenges
- Silent failures. A change that doesn't crash can still give wrong numbers, so we compare results with a baseline instead of only catching errors.
- Real-world columns. A nested (struct) column broke our first type check, so complex types are now skipped safely.
- Free Edition limits. Databricks Apps can't be opened by people outside the workspace, so we published a public copy of the interface.
- Trusting the model. The model never decides if a deal is valid. Rules do.
What we learned
Letting the model write and propose while rules referee keeps the result checkable. Measuring impact with real queries beats guessing from lineage alone. And real data has surprises, so a sandbox that tests every change finds problems early.
Honest limits
The data is Databricks' simulated sample data, and the reports are generated from the schema. The AI negotiations were computed in advance, and the public page replays them. Four kinds of change are measured.
