Inspiration
I've been working at Tasteport, a grocery delivery startup in Toronto. We're starting to run some digital inventory operations, and I've spent a lot of my time fixing databases that quietly break. A vendor retypes one thing in one column, usually no error :(, and six hours later the store owner is looking at the wrong #s on the dashboard and if it's big it's usually on me to explain why. I feel like this plagues a lot of engineers working with databases so I wanted to create a solution.
What it does
① signal schema fingerprint diff + failing assertions ② recall search DataHub documents for prior incidents on this asset ③ blast radius walk column-level lineage; score impact; find critical paths ④ diagnose LLM investigates root cause with DataHub tools bound ⑤ plan declarative policy decides autonomy: auto-apply / PR / escalate ⑥ generate render dbt model + contract + test + assertion from live schema ⑦ verify parse SQL, check every column against DataHub, prove contract holds ⑧ gate human approval via a real LangGraph interrupt ⑨ remember write postmortem, tags and column docs back into DataHub
How we built it
Catalog interface with two implementations: The fixture catalog is the same warehouse in YAML (offline, zero credentials — suture demo works anywhere); the live catalog drives a real DataHub instance through the DataHub Agent Context Kit's tool surface (search, schema fields, lineage, assertions, documents, tags, descriptions). Same agent, same maths, byte-for-byte identical scores in both. The agent itself is a LangGraph state machine (sense → recall → blast_radius → diagnose → plan → generate ⇄ verify → gate → act → remember). Verification failures loop back into generation with the error text, bounded retries — nothing touches disk until it survives every check.
Challenges we ran into
- Column-level lineage hiding: Core's graph APIs stopped returning fineGrainedLineages; without them every change looked like it hit everything downstream. Fix: read the upstreamLineage aspect directly from GMS. Changing that is why amount-retyped (7 assets, CRITICAL) and status-dropped (1 asset, LOW) get opposite answers on the same table.
- "No drift detected" was sometimes a lie. The search index is eventually consistent and survives restarts with stale state — after a DataHub restart we learned a drifted schema as "baseline". Fix: read schemas from the stored schemaMetadata aspect, deterministic with the write.
- Writebacks fight back. Creating tags, describing columns, saving documents — each surfaced a real DataHub quirk (tag entities must exist first; removed columns can't be documented; kit argument names differ from docs). Every one got fixed and recorded, never silently dropped. ## Accomplishments that we're proud of
- The contrast: same table, two changes, opposite responses — CRITICAL/ESCALATE vs LOW/PROPOSE_PR — decided by column-level lineage, live, in front of you.
- Real gate. A repair that needs a human genuinely stops the machine, survives a crash via sqlite checkpoints, and no file is written without approval — there's a test that kills the process mid-gate and resumes.
- Nothing unverified ships. Generated SQL is parsed, every identifier checked against the live schema, and the downstream contract preserved to the exact type — DECIMAL(18) is not DECIMAL(18,2). dbt compile runs on the exact files the agent generated, in CI. ## What we learned Column-level lineage is the entire difference between a useful alert and noise — "everything downstream". Deterministic reads beat eventually-consistent reads for anything that triggers alarms. ## What's next for Suture Ship the repaired contract as DataHub assertions that monitor the fixed columns — so the next regression fails in the catalog, not in a dashboard. "try it out on your own repo" feature.
Log in or sign up for Devpost to join the conversation.