Kavach — Self-Healing Data Platform

AI agents on DataHub’s context graph detect, diagnose, fix, and learn from data incidents — health for systems and health for humans.

Comment

Inspiration

When a pipeline breaks at 2am, you still bounce between tabs — lineage in one place, ownership in another, a postmortem that never gets written. A bad mart column can poison a production model. In healthcare, a leaked PHI column or a null medication code is not a dashboard miss — it is a patient-safety miss.

Kavach is one agent team on DataHub’s context graph for both stakes: health for systems (retail / ML) and health for humans (PHI / clinical quality).

What it does

Pick a real failure, inject chaos, and watch the war room respond:

  • Detect — Sentinel confirms the anomaly via DataHub assertions and opens an incident
  • Diagnose — Investigator and Impact Analyst walk lineage and query history
  • Protect ML — ML Guardian can hold a deployment when features look poisoned
  • Fix — Fixer drafts a dbt PR with safeguard tests
  • Learn — Scribe writes the postmortem back as a Context Document so the next run is faster

Judges can use Replay with zero API keys. Toggle Systems | Humans in the header for retail vs Synthea-style healthcare scenarios.

How we built it

  • FastAPI + LangGraph for a 7-agent team, with a chaos engine that seeds deterministic failures
  • DuckDB + dbt for the retail pipeline; scikit-learn demand forecast with ML lineage in DataHub
  • Next.js war room (agent feed, blast radius, Atlas “How Kavach works”) plus /deck for the pitch
  • Real DataHub MCP (JSON-RPC / Streamable HTTP) and Agent Context Kit tools when a GMS is live; otherwise fixtures + committed recordings
  • Deployed frontend on Vercel: https://kavach-self.vercel.app

Challenges we ran into

Speaking real MCP mattered. An early client hit fake /mcp/tools paths; we rewrote it to initialize + tools/call over JSON-RPC. We also labeled LIVE / REPLAY / DEMO honestly so judges never confuse fixture demos with live GMS. Mapping column-level lineage through features into an ML deployment hold was the hard part of the blast-radius story.

Accomplishments

  • End-to-end incident loop with write-back to DataHub (incidents, tags, glossary, Context Documents)
  • Two domains with correct data sources: NYC TLC for Systems, Synthea (no real PHI) for Humans
  • Merged Fixer PR on the demo pipeline, plus dry-run PR artifacts for other scenarios
  • OSS skill opened upstream: https://github.com/datahub-project/datahub-skills/pull/61
  • Judge-facing examples/ — PRs, postmortems, assertions, redacted MCP transcripts

What we learned

Context compounds. When the postmortem lands back in DataHub, the next agent run starts with evidence instead of a blank slate. That is the flywheel — not a prettier alert.

What's next

  • DataHub Cloud trial only on Aug 10–11 so the 21 days cover judging
  • Merge the skill PR when a maintainer approves
  • Optional Ask DataHub beat once Cloud is up

Built With

Share this project:

Updates