Inspiration
Every wrongfully denied health insurance claim is a real person's delayed care — and the cause is often invisible: a silent data pipeline defect nobody notices for weeks. Meanwhile, CMS-0057-F is forcing every U.S. payer toward FHIR-based interoperability, with prior-authorization metrics reporting already mandatory since January 2026. We wanted to know: could a team of agents catch these defects automatically, prove where they actually came from instead of reflexively blaming "the data," and fix them — using DataHub as the shared source of truth the whole team reasons over?
What it does
Claims Denial Guardian is a multi-agent pipeline that watches a healthcare claims system end to end:
- Sentinel statistically flags abnormal denial patterns (a two-proportion z-test, zero LLM — deterministic, auditable, recomputable by hand)
- Investigator — an LLM agent reading DataHub's lineage graph through its MCP Server — traces each anomaly to a real root cause, discriminating between defects introduced at the claims layer versus inherited from upstream source data
- Remediator generates a fix as real, schema-aware SQL and opens an actual pull request against the data platform's repo — using a quarantine-not-correct philosophy: suspect rows are set aside for review, never silently "corrected"
- Scribe writes every finding back into DataHub itself — tags, documentation, assertions — so the next engineer or agent inherits the knowledge instead of starting from zero
- Reporter produces compliance-ready audit reports (Markdown + self-contained HTML)
- A feature-health check monitors the registered ML model's inputs via DataHub's ML lineage
- A FHIR bridge generates real CMS-relevant FHIR R4 resources, each flagged with exactly which incident implicates the data it's built on
Two seeded incidents prove the system actually discriminates rather than pattern-matches: one defect injected directly into claims, one injected further upstream into raw source data. Same detector, structurally different, correctly-traced answers — verified live against the real DataHub graph.
How we built it
Python + the Anthropic SDK, talking to DataHub entirely through its MCP Server for reads and its SDK for writes. Three pluggable LLM backends (an Anthropic API key, a Claude Pro subscription via headless Claude Code, or fully local/free via Ollama) so anyone can run it, regardless of budget. SQLite holds the claims/denials data, built on top of DataHub's own healthcare sample dataset. A second small repo, denial-guardian-data-platform, simulates the team that owns the pipeline — so the Remediator's pull requests are real, reviewable artifacts against an actual codebase, not a self-directed diff.
All four hackathon tracks, concretely:
- Agents That Do Real Work — Sentinel/Investigator/Scribe read and write DataHub's context graph live, no mocking
- Metadata-Aware Code Generation — the Remediator reads real, live schemas before generating fix code; PR #1 and PR #2 are real, merge-ready pull requests
- Production ML Agents — a feature-health check against the registered
MLModel/MLFeature/MLFeatureTableentities - Open/Wildcard — the FHIR compliance bridge, directly targeting CMS-0057-F's interoperability push
Challenges we ran into
The honest ones taught us the most. Live testing repeatedly caught things mocked tests couldn't: a DataHub MCP wrapper silently reshaping fields, an aspect-overwrite bug where re-running ingestion clobbered custom metadata patches, and — most seriously — discovering that DataHub's SDK has no supported mechanism for representing an MLModel as a lineage upstream of a Dataset, something we confirmed via three independent exhausted approaches and reported upstream. We also caught our own mistakes along the way: an early root-cause finding turned out to be based on a flawed re-test, corrected in place rather than left standing once we found the real answer.
We also drew a hard line on honesty in the artifacts themselves: the FHIR resources use FHIR's own data-absent-reason mechanism for fields we genuinely can't populate, rather than inventing plausible-looking ICD-10 codes. If we didn't know something, the system says so — it doesn't guess.
Accomplishments that we're proud of
A fully working, end-to-end agentic pipeline — detection through remediation through compliance export — verified live against a real DataHub instance, not simulated. Two upstream contributions filed against datahub-project/datahub (#18742, #18743) with full, reproducible evidence. And an adversarial proof that nothing is hardcoded: relocate the seeded anomaly to a different segment, and the same statistical detector finds it there instead — behavior, not code review, as the evidence.
What we learned
That the deepest bugs hide exactly where you stop looking — in the gap between "the mocked test passed" and "it actually works against a live system." And that an agent team is genuinely more trustworthy when it's designed to say "I don't know" or "this needs your approval" rather than filling every gap with a plausible-sounding guess.
What's next for Claims Denial Guardian
Real ICD-10/CARC code mapping and Da Vinci PAS/PDex profile conformance for the FHIR bridge; a scheduled/continuous version of the drift check; expanding the Remediator's fix vocabulary beyond the quarantine pattern to cover more defect classes.
Log in or sign up for Devpost to join the conversation.