SchemaShield

Inspiration

A schema change can be valid in isolation and still break models, dashboards, queries, or ML consumers downstream. A rename such as order_total to gross_amount looks small in a pull request, while downstream transformations may still reference the old field. SchemaShield adds catalog evidence and a durable decision trail before merge.

What it does

SchemaShield has two explicit execution surfaces.

The qualifying local path runs a scoped synthetic round trip against DataHub OSS. It seeds three DEV datasets, reads the registered schema through Agent Context Kit, reads two-hop downstream lineage through the DataHub Python SDK, and requires an explicit approval flag before applying a risk tag and saving a decision document. A sanitized JSONL trace records the version, synthetic entities, schema fields, lineage hops, tool boundary, and writeback identifiers without credentials.

The complementary hosted path is a deterministic offline replay. Judges can inspect three bounded cases without credentials:

• a nullable field addition, classified LOW;
• a revenue field rename, classified HIGH;
• a lossy decimal-to-integer change with a production ML dependency, classified CRITICAL and blocked.

Each run exposes the decision, evidence references, blast radius, affected queries, generated compatibility SQL, and a deterministic run ID. The page visibly reports that it is an OFFLINE SNAPSHOT with zero live writebacks.

How we built it

The reference local run used DataHub OSS v1.6.0, datahub-agent-context==1.6.0.17, and acryl-datahub==1.6.0.6. Agent Context Kit read the registered fields and performed the approved tag, document, and entity retrieval operations. The SDK returned the two-hop lineage for the tested version pairing.

The deterministic core is JavaScript. It canonicalizes inputs, derives SHA-256 run IDs, allowlists supported operations, validates SQL identifiers, requires an explicit lossy-change flag, rejects ungrounded rename targets, and refuses accidental artifact overwrites. It generates reviewable SQL, dbt YAML, an impact report, PR summary, provenance, submission status, and an unapplied writeback plan.

The public interface uses Next.js 16, React 19, TypeScript, and Vercel. The source is Apache-2.0 licensed and includes automated tests, committed synthetic fixtures, reproducible examples, the sanitized trace, and a claims-boundary document.

Challenges we ran into

The hardest product decision was making a useful public demo without overstating integration. A hosted browser that silently pretended to be connected to DataHub would be misleading, so SchemaShield labels the hosted path as an offline replay and treats the reproducible local DataHub trace as separate evidence.

We also found version-specific lineage behavior: Agent Context Kit read the schema and performed tag/document operations, but its lineage helper did not return the already indexed downstream graph in the tested pairing. The DataHub SDK and CLI did. We used the narrow SDK fallback and documented the boundary instead of relabeling the result.

Finally, generated migration code should not be accepted merely because it is syntactically plausible. The core therefore fails closed on ambiguous or unsupported inputs.

Accomplishments that we're proud of

• Completed a real synthetic local DataHub read/write round trip.
• Made the exact Agent Context Kit and SDK boundary visible.
• Built a free, no-login replay with three materially different decisions.
• Generated inspectable compatibility and review artifacts.
• Preserved deterministic provenance and a sanitized evidence trace.
• Added fail-closed checks for unsupported operations and unsafe identifiers.

These are implementation and replay accomplishments, not claims of production readiness, adoption, accuracy, savings, or business impact.

What we learned

Metadata is most useful to an agent when it supports both sides of the loop: reading context before acting and writing an approved decision back afterward. Tool provenance matters as much as the observation itself. Deterministic outputs also make generated code easier for reviewers to trust and reproduce.

What's next

We plan to feed the live DataHub response directly into the artifact generator, add a human-approved Git provider integration, support warehouse-specific SQL validation, expand the bounded operation set, and test authenticated remote deployments separately.

Try it

• Live demo: https://schema-shield.vercel.app
• Source and evidence: https://github.com/ceodaradigu/schema-shield

Built With

• agent
• datahub
• javascript
• nextjs
• python
• react
• typescript
• vercel