HealthCare-Sentinel

Healthcare Sentinel is an AI agent that triages healthcare data quality like a clinician — discovers issues in DataHub, explains patient harm, remediates with approval, and learns across runs.

Comment

Inspiration

I build and maintain a health and fitness app. Every day I think about how data affects real people's lives. When a user logs a workout or tracks their vitals, bad data doesn't just look wrong on a dashboard. It changes the advice they get. It changes their outcomes.

With my inclination towards healthcare, when I saw the DataHub hackathon's sample healthcare dataset (55,500 patient records with planted quality issues), I knew this was the challenge I wanted to take on. The dataset has real problems: patients with age negative 43, 555 records with no patient name, 277 patients discharged before they were admitted, 1,215 negative billing amounts.

Traditional data quality tools would flag these as "832 rows out of range." But in healthcare, that number means something specific. It means drug dosing errors. It means patients who can't be identified in an emergency. I wanted to build something that understands that difference. Something that thinks like a clinician, not a script. That's Healthcare Sentinel.

What it does

Healthcare Sentinel is an AI agent that connects to DataHub, autonomously discovers healthcare datasets, and triages data quality issues with clinical reasoning.

No hardcoded rules. No predefined checks. No hints about what to look for.

The agent figures it out on its own:

  1. Discovers datasets by searching the DataHub catalog
  2. Reads schemas and generates its own SQL quality checks
  3. Runs the checks and samples bad rows for evidence
  4. Traces lineage downstream to find contaminated tables
  5. Ranks by clinical severity, not just row counts, but actual patient harm
  6. Writes back to DataHub with severity tags and warnings
  7. Fixes the data with your approval (correct, quarantine, flag)
  8. Saves what it learned so the next run starts smarter
  9. Verifies independently with a second reviewer agent that re-runs every query

Every data fix is captured with row-level CDC (change data capture). Every operation is fully reversible with --undo. Even denied proposals get logged to the audit trail.

The next person who opens that dataset in DataHub sees the warnings. Knowledge persists across people, not just across runs.

How we built it

Agent core with LangGraph The entire system is a single Python file (sentinel.py, ~2,200 lines) built on LangGraph's create_react_agent. We added a human-in-the-loop middleware that batches all data mutation proposals into one approval prompt. The agent gets 12 tools total: 9 from DataHub Agent Context Kit plus three custom ones (run_sql, report_finding, and apply_fix). We chose LangGraph specifically because its interrupt/resume model made it straightforward to pause the agent mid-run for human approval and continue exactly where it left off.

DataHub Agent Context Kit as the backbone We use 9 of the 10 available tools. search for dataset discovery. list_schema_fields for schema inspection. get_lineage for downstream contamination tracing. add_tags and update_description for writing clinical severity back. save_document and search_documents for the self-learning memory system. get_entities for metadata enrichment. get_dataset_queries for query pattern analysis. The Context Kit gave us a clean interface to the entire DataHub catalog without writing any GraphQL or REST calls ourselves.

Two-pass triage and review Pass 1 is the full triage scan where the agent discovers, investigates, and remediates. Pass 2 is an independent reviewer that re-runs every SQL query to confirm findings, checks counter-claims (could that negative billing be a valid refund?), and runs its own investigation for anything the first pass missed. The reviewer operates on a pre-fix database snapshot so it always verifies against the original data. This catches confirmation bias from the first pass.

Self-learning with compound knowledge The agent saves what it learned as a DataHub document after each run. On the next run, it recalls those learnings, verifies which issues are still live by re-running the evidence queries, and then continues investigating. When new data arrives (like a second clinic's records), the agent discovers the new table through sqlite_master, applies its prior knowledge, and finds new issues specific to that dataset. Each run genuinely builds on the last.

Human-in-the-loop remediation Metadata writes (tags, descriptions) are automatic because they don't change patient data. Data mutations (UPDATE, DELETE, INSERT) require explicit human approval. The agent proposes fixes in batches. You see the SQL, the affected row count, and the clinical reasoning. You approve all, pick specific ones, or deny everything. Every decision gets logged with the reviewer identity, hostname, and timestamp.

Row-level CDC and full audit trail Before any data fix, the system captures a before-image of every affected row in a changelog table. Every operation gets a unique ID, and you can undo any fix with python sentinel.py --undo N. The audit trail records both approved and denied operations, making the entire remediation history queryable and reversible.

Multi-model support Works with both Gemini 2.5 Pro (via Vertex AI) and Claude. The system prompt and tool design are model-agnostic. We tested primarily with Gemini for the hackathon since it performed well with parallel tool calls and clinical reasoning.

Challenges we ran into

  • Getting the agent to not hardcode anything. Early versions had the agent memorizing table names and check patterns from previous runs. We rewrote the system prompt multiple times to enforce evidence-first investigation. The agent now discovers everything from DataHub metadata and sqlite_master. No hints, no static patterns.

  • Finding count stability. LLM-based agents are non-deterministic. The same data, same prompt, different run can produce 5 findings or 9 findings. We addressed this with the two-pass reviewer architecture and a collapse algorithm that merges downstream findings into source findings. But some variance is inherent to the approach. We learned to be honest about that rather than hide it.

  • Learnings pipeline. Getting the agent to properly save, recall, and verify learnings across runs took several iterations. The DataHub document API creates new documents by default, so we built an upsert wrapper that reuses existing document URNs. Clearing learnings on reset required hard_delete_entity since the standard delete mutations didn't apply to Document entities.

  • Downstream finding collapse. When the agent finds "invalid age" in raw_patients and "inherited invalid age" in staging_patients and mart_demographics, those are one issue, not three. But the agent uses different check names each run. We built a substring-matching collapse that merges downstream findings into source findings based on check name hierarchy and column matching, without hardcoding any patterns.

  • Quarantine table creation. The agent sometimes tries to INSERT into a quarantine table before creating it. We didn't want to hardcode a workaround. Instead the agent learned to self-heal by falling back to a flag column approach when quarantine fails, then retrying with CREATE TABLE IF NOT EXISTS on the next attempt.

Accomplishments that we're proud of

  • Zero hardcoding. The agent discovers every dataset, generates every check, and reasons about every finding from scratch. Swap the database and it works on completely different data.

  • Compound learning actually works. Run 1 finds issues, fixes them, saves learnings. Add new data. Run 2 recalls learnings, verifies fixes, discovers new tables, finds new issues. Each run genuinely builds on the last.

  • Every finding is real. Across dozens of test runs, zero false positives. When the agent says "832 impossible ages," there are exactly 832 impossible ages in the database.

  • Clinical severity reasoning. The agent doesn't just flag "832 rows out of range." It explains that impossible ages cause fatal drug dosing errors, that missing names make patients invisible in the ER, that negative billing amounts corrupt revenue accounting.

  • Full reversibility. Row-level CDC on every fix. Undo any operation. Audit trail with forensic device context. This is the kind of safety you need for healthcare data.

What we learned

AI agents are powerful but not deterministic. The same agent, same data, same prompt can investigate deeply one run and skip things the next. That's not a bug you can fix with better prompting. It's a fundamental property of LLM-based reasoning.

The right response isn't to pretend the agent is perfect. It's to build safety nets around it. A second reviewer pass, human approval gates, row-level undo, and transparent audit trails. The system is designed so that when the agent misses something, the human catches it. And when the human misses something, the next run's learnings fill the gap.

Building on DataHub Agent Context Kit was a great experience. Having 10 well-designed tools for catalog interaction meant we could focus on the clinical reasoning and remediation logic rather than building metadata plumbing from scratch. The save_document and search_documents tools made the self-learning system possible without any external storage.

What's next for HealthCare-Sentinel

  • File watcher mode is already built. The --watch flag monitors a directory for new CSV, Excel, or SQLite files and triggers a triage run automatically when data arrives.

  • Multi-database support. Extend beyond SQLite to PostgreSQL, BigQuery, and Snowflake for production healthcare data warehouses.

  • Structured properties. Use DataHub's structured properties to store finding metadata (severity, affected rows, clinical impact) as machine-readable annotations, not just description text.

  • Health and fitness data. Apply the same data quality agent to fitness and health tracking data. Bad sensor readings, impossible heart rates, and missing workout logs affect real people's health decisions just like bad clinical data affects patients.

Built With

  • agent
  • ai
  • claude
  • cli
  • datahub
  • datahub-agent-context-kit
  • docker
  • google-gemini
  • langchain
  • langgraph
  • llm
  • python
  • sqlite
  • vertex-ai
Share this project:

Updates