GreenLight

• 

• 

• 

Inspiration

Lineage tells you what a data change might break. It rarely tells you whether the change is actually ready to ship.

That decision also depends on ownership, compliance classifications, escalation contacts, freshness commitments, and the people responsible for downstream assets. The information already exists in DataHub, but change workflows usually read only the lineage portion of the graph.

greenlight brings those signals together to answer a practical question: What has to happen before this table change can ship?

What it does

greenlight turns a proposed table change into one release decision:

• whether a governance policy blocks it;
• who must approve it;
• who must be notified;
• which downstream assets have no owner recorded; and
• which conditions, such as freshness or retention commitments, still apply.

A user can provide a specific table with --target or ask a normal business question with --ask. When several assets share a name, greenlight first uses DataHub declarations and five equally weighted catalog signals to select the intended asset. There are no tunable coefficients. Signals can abstain, and if the graph cannot separate the candidates, greenlight refuses to choose.

The filmed example asks which table is order_details. DataHub identifies five candidates and declares one canonical asset. Changing it reaches 24 downstream assets, requires 3 sign-offs from the Data Platform Team, DataHub SE Team, and Sarah Chen, creates 9 notification obligations, and is blocked by GDPR and SOC2. It also reports that 16 impacted assets have no owner recorded rather than inventing contacts for them.

One release decision from DataHub context: blocked by GDPR and SOC2, with 3 signers, 9 notification obligations, and 16 ownership gaps across 24 downstream assets.

Every conclusion carries the exact DataHub aspect, URN, rule, and literal value behind it. A reviewer can inspect the evidence instead of trusting a score. The resolver is deliberately deterministic: there is no model in the decision path. greenlight serves the same result over CLI, a self-contained HTML report, and three MCP tools so a person or an agent can check readiness before opening a pull request.

Every conclusion is auditable. Selecting the GDPR blocker reveals the exact DataHub glossaryTerms aspect, rule, literal value, and URN behind the decision.

In live mode, --emit can write the review back as namespaced structured properties. When selection succeeds, superseded candidates receive a deprecation whose replacement points to the winner. A refused selection writes no deprecation, and fixture mode never writes to DataHub.

How we built it

A proposed table change and the governance metadata already in DataHub become one release decision: blockers, approvers, notifications, and ownership gaps, with every conclusion traced to its source aspect.

View the full technical architecture.

One catalog enters through client.py: either the bundled, unmodified showcase-ecommerce datapack or a live DataHub instance. In live mode, mcp-server-datahub performs discovery and name resolution. DataHub's OpenAPI entity endpoint supplies the aspects required for ownership, glossary terms, lineage, SLAs, retention, and write-back.

select.py determines which asset the user meant. resolve.py derives the release obligations. Both return the same Chain domain object used by the CLI, HTML dashboard, MCP server, evaluation harness, and mutation payload, so those surfaces cannot quietly disagree.

The implementation uses Python 3.14, Poetry, DataHub OSS v1.7.0, the DataHub SDK, mcp-server-datahub over stdio, an MCP server of its own, pytest, ruff, Playwright, and a zero-runtime-dependency HTML report.

Challenges we ran into

• DataHub's datapack wraps aspects as GenericAspect JSON strings. Reading the wrong layer made governed assets look empty, so aspect reads moved to the OpenAPI entity endpoint.
• The MCP search endpoint caps results at 50, while graph hydration became unreliable after several URNs. We measured the boundary and use MCP for the discovery work it handles reliably.
• Counts are not citations. Selection originally cited totals such as "fourteen downstream assets," which could not be traced to one source fact. The evaluator now requires evidence from the actual consuming asset, ownership record, SLA, or other contributing aspect.
• A refusal must not look like a decision. Tied candidates now route to their owners without producing a winner or retirement plan.

Accomplishments that we're proud of

The evaluation is not limited to the demo table. It sweeps all 67 datasets in DataHub's published catalog and traces 1,161 of 1,161 routing citations, with zero unbacked. It also reports 379 of 379 unowned downstream gaps.

Selection is evaluated across all 15 ambiguous business names: 14 are selected, one is refused, and 41 of 41 selection citations trace back to their source aspects.

Five falsification checks then remove glossary terms, ownership, escalation, certification, or every separating signal. In every case the dependent conclusion disappears or greenlight refuses to rule. The implementation is also covered by 35 Python tests and 14 browser tests, including keyboard navigation, no-JavaScript evidence, refusal behavior, and desktop/mobile overflow.

The evaluator sweeps all 67 catalog datasets and all 15 ambiguous names: 1,161/1,161 routing citations and 41/41 selection citations traced, with zero unbacked evidence.

What we learned

The most useful output is not always the approval list. Sometimes it is the gap: the downstream asset with no owner, the ranking signal that abstained, or the question the catalog could not answer. Publishing those limitations makes an agent safer because silence can no longer look like certainty.

We also learned that more evidence is often better than a looser threshold. Reading DataHub's siblings aspect reduced false refusals without weakening any selection rule.

What's next for greenlight

• Post the release chain as a pull-request check and request review from the resolved approvers.
• Emit the decision as a native DataHub Incident.
• Use an escalation contact's manager when an asset owner is inactive.

Built With

• datahub
• docker
• mcp
• python