Zero-Copy Sentinel

• 

  Overview

• 

  Architecture

• 

  Data Assets

• 

  Incident

• 

  Lineage

Inspiration

Zero-copy means moving or using data without making extra copies of it and it cuts latency, storage, and compute costs, and removes pipeline complexity. Major SaaS and enterprise vendors are adopting zero-copy architectures so customers can share and query data directly, without traditional ETL. A shared open metadata layer based on Apache Iceberg that lets AI agents and applications query fresh enterprise data instantly, with zero lag. Zero-copy data sharing is incredibly powerful, but it introduces a subtle problem most teams never detect. Zero-copy has no pipelines and when a share goes stale or a source schema changes, nothing fails, nothing alerts, and no one notices until a number is wrong in an executive dashboard. Data drifts doesn't error, it just goes unnoticed whether it's a stale value, a dropped column, or an ungoverned copy someone materialized for performance.

The fix: Zero-copy sharing breaks the old way of inferring lineage from pipeline runs so the fix has to be a metadata platform where shares themselves are first-class lineage, which is exactly what DataHub's model supports. This agent reads and writes through that graph directly: catching drift, raising a real Incident, and putting a human in the loop instead of auto-remediating

What it does

A governance agent (LangGraph + DataHub's Agent Context Kit, powered by Claude) that reads DataHub's lineage graph, detects two kinds of silent drift between a zero-copy source and a materialized copy stale values and dropped-column schema breaks and writes each finding back to DataHub (a tag + description, or a real Incident) and to Slack.

How we built it

• agent.py — the skeleton: connects to DataHub, loads the Agent Context Kit's tools (build_langchain_tools) plus this project's custom tools, wires a LangGraph create_react_agent, and runs - the goal — streaming each tool call and result to the terminal as it happens, so you watch the agent think instead of waiting for a final report.
• goal.py — the sentinel's actual instructions: a SYSTEM_PROMPT (read before you act, be conservative, report explicitly even when nothing's wrong) and a GOAL that runs both drift checks.
• custom_tools.py — two tools the Agent Context Kit doesn't ship:
• raise_incident — wraps DataHub's raiseIncident GraphQL mutation (confirmed via live schema introspection, not assumed from docs).
• notify_slack — posts to an incoming webhook, reading SLACK_WEBHOOK_URL from .env.
• seed_zero_copy_demo.py — seeds the demo scenario. Two modes: --mode seeded (default) — fabricates both datasets locally, no external dependencies. --mode snowflake — enriches a real, already-ingested Snowflake table instead of fabricating one.
• snowflake/ — SQL to set up a real Snowflake table (REVENUE_LIVE_ICEBERG), add a discount_pct column, then drop it later to trigger genuine schema drift on a real warehouse. snowflake_ingest.yml — DataHub ingestion recipe for pulling that table in. Lineage/usage extraction is disabled because it needs SNOWFLAKE.ACCOUNT_USAGE access the ingest role doesn't have (lineage is set manually via the SDK instead).
• ingest.sh — wrapper that sources .env before running datahub ingest. The raw command fails with expandvars.UnboundVariable in a plain terminal because the recipe's ${SNOWFLAKE_*} placeholders resolve from real environment variables — use this instead.
• reset_demo.py — resets the demo dataset to its pristine state (removes the stale-copy-risk tag, restores the description, hard-deletes incidents). Run it between demo takes — the agent's write-backs are real, so a dry run dirties the catalog.
• Every write-back in this project has been independently confirmed against DataHub's raw GraphQL API — not just the agent's own self-report. That includes catching a real bug along the way: re-running datahub ingest after seeding the zero-copy properties silently wiped them (DataHub aspects are last-writer-wins, not merged) — caught because the agent correctly reported a missing property instead of hallucinating a number.

Challenges we ran into

1. The plumbing is the first real blocker, not the agent logic. Before any agent code ran, there was a Python version conflict — 3.14 had no prebuilt wheel for pydantic-core and no Rust toolchain to build one, forcing a drop to 3.12 — plus a shell alias silently shadowing the venv's python. Neither had anything to do with AI; both cost real time. The lesson: budget for environment friction as its own phase, not a footnote before "the real work."

2. Permission models hide in layers — one fix rarely closes the issue. The Snowflake ACCOUNT_USAGE error came back three separate times, each fix (a grant, then use_queries_v2, then include_operational_stats) only revealing the next layer underneath it. The habit this forced: after every fix, re-run and re-check — don't assume the second error means the first fix failed.

3. Silence is the most dangerous failure mode, and I hit it from every angle. Ingestion silently wiping seeded properties, a script with zero output until it finished, a UI preview quietly showing stale data, a lineage graph that simply stopped drawing a broken connection instead of flagging it — none of these threw an error. That's not a coincidence relative to the project's actual thesis; it's the same lesson learned twice, once by design and once by living it.

4. Verify against the live system — a report is not the system. Screenshots, a script's own "success" message, an exit code of 0 — none were reliable on their own. The only thing that consistently caught real problems was checking the actual API/database state independently. This stopped being a one-off correction and became the default way of working.

5. Least-privilege boundaries are supposed to be inconvenient. Getting blocked from patching data directly because the ingestion role only had SELECT wasn't a bug to work around — it was the access model doing exactly its job. Worth remembering the annoyance is often the point.

Accomplishments that we're proud of

1. Shipped a fully working dual-mode agent : value drift and schema drift with real write-back to DataHub (tags, rewritten descriptions, a genuine Incident) and Slack, not a mockup or a slide deck of what it would do.
2. Verified end-to-end against a real Snowflake account: real transaction data, a real ALTER TABLE ADD/DROP COLUMN to trigger genuine schema drift, not a simulated metadata fixture pretending to be one.
3. The agent discriminates correctly: out of 4 dashboards inspected in its last verified run, it flagged exactly the one with real drift and left the other three alone zero false positives. Severity-matched response: a tag + description for "worth noticing," a real HIGH-priority Incident for "someone needs to act" not one flat alert for everything.
4. Caught a moment of genuine model honesty in production: when the live source's comparison property was missing, the agent said so explicitly rather than fabricating a number.
5. Built on DataHub's actual official starter pattern (LangGraph + Agent Context Kit) rather than a one-off hack something forkable, not a dead end.

What we learned

Running the official starter agent end-to-end, before customizing anything, taught more about how the pieces (LangGraph, the Agent Context Kit, Claude's tool-calling loop) actually fit together than any documentation would have. Environment friction is the first real blocker, not the AI logic a Python version conflict and a shell alias silently shadowing a venv cost real time before any agent code ran.

What's next for Zero-Copy Sentinel

Phantom-copy detection: the third failure mode: an ungoverned copy with no zero-copy/materialized-copy tag pairing at all, invisible to both current checks.

Propose, don't just detect: have the agent suggest a remediation (e.g., "point the dashboard at the live table instead") while keeping a human as the one who applies it.

Use real iceberg table in a production environment and detect more failure modes. We used mock iceberg tables not actual ones.

Built With

• claudecode
• datahub
• langchain