LineageGuard

Inspiration

Schema changes are reviewed inside a code repository, while their real consequences cross warehouses, transformation projects, dashboards, and machine-learning features. LineageGuard was built around the gap between what a pull request can see and what organizational metadata already knows.

What it does

LineageGuard accepts a proposed schema change, retrieves governed context and column-level lineage from DataHub, and returns an ALLOW, WARN, BLOCK, or ABSTAIN decision with explicit confidence and evidence paths. For the demo rename, it discovers three downstream paths, identifies contract-governed assets, produces two reviewable repair patches, routes the review to affected owners, and writes an approved mitigation record back to the source demo asset.

How it was built

• Typed Python decision core with deterministic safety gates.
• DataHub Python SDK for dataset creation, schema and ownership metadata, fine-grained lineage traversal, and custom-property write-back.
• Evidence-first local web interface served by Python's standard library.
• Project-authored synthetic metadata and repository fixtures.
• Static, explicitly read-only judge preview for environments without a DataHub backend.

Challenges

The official showcase Datapack contains several aspects rejected by the tested stable OSS server despite its published compatibility metadata. Rather than silently discard arbitrary records, the project keeps an auditable importer diagnostic and uses a smaller project-authored graph composed only of supported standard SDK entities and lineage.

Accomplishments

• Live read from DataHub to a non-trivial merge decision.
• Three inspectable downstream paths across analytics, executive reporting, and an ML feature.
• Two evidence-linked repair artifacts.
• Human-gated write-back with read-after-write verification.
• Clean-clone reproduction, 14 tests, dependency QA, and zero external cost.

What we learned

Metadata becomes far more useful when it changes an engineering action rather than merely answering a catalog question. The strongest boundary is also the most important: missing context must produce abstention, not confident invention.

What's next

Add pull-request provider adapters, richer schema-diff parsing, transactional patch branches, and production authorization policies. The current build is intentionally scoped to a synthetic golden path and does not claim universal migration coverage.

Originality and provenance

The project was created during the competition submission period. Standard frameworks, libraries, starter resources, and coding tools are handled according to the official rules; any other pre-existing work would be identified in the final submission if incorporated.

Built With

• css
• datahub
• datahub-sdk
• docker
• html
• javascript
• python