-
-
Live incident response dashboard with DataHub connectivity and system health.
-
AI investigates a production data quality incident using DataHub lineage.
-
Lineage-aware blast radius analysis identifies only affected downstream assets.
-
Verified incident metadata written back into DataHub after human approval.
Inspiration
What it does
How we built it
Challenges we ran into
Accomplish# Inspiration
Machine learning systems rarely fail because a model suddenly stops working. More often, failures begin with silent data quality issues that propagate through datasets, feature pipelines, models, and production endpoints. By the time engineers notice the problem, identifying the root cause can take hours.
We built LineageMedic to reduce that investigation from hours to minutes by combining DataHub's metadata graph with AI-assisted incident response.
What it does
LineageMedic is an AI-powered incident response platform built on DataHub.
When a production data quality issue is detected, it:
- Retrieves live metadata and lineage from DataHub
- Identifies the likely root cause
- Calculates the downstream blast radius
- Generates a safe remediation plan
- Requires explicit human approval before changes
- Writes verified incident information back into DataHub
Instead of only raising alerts, LineageMedic helps engineers understand why an incident happened, what is affected, and how to respond safely.
How we built it
The platform combines multiple components into a single workflow:
- DataHub OSS as the metadata platform
- DataHub GraphQL and MCP integration
- FastAPI backend
- React frontend
- Python-based orchestration
- Multi-agent workflow for investigation and remediation
Each investigation uses live metadata from DataHub instead of static configuration, allowing decisions to be grounded in the current state of the platform.
Challenges we ran into
The biggest challenge was integrating real-time metadata retrieval, lineage analysis, AI reasoning, and DataHub writeback into one reliable workflow.
We also focused on making remediation safe by introducing a mandatory human approval step before any metadata changes are written back.
Accomplishments that we're proud of
- End-to-end incident investigation
- Live DataHub metadata integration
- Lineage-aware blast radius analysis
- Human-approved remediation workflow
- Verified metadata writeback to DataHub
- Production-ready REST API
What we learned
This project reinforced that metadata becomes significantly more valuable when it actively supports operational decision-making.
Rather than serving only as documentation, DataHub can become the foundation of intelligent incident response.
What's next for LineageMedic
Future work includes:
- Support for additional DataHub metadata types
- Automated policy-driven remediation
- Richer incident analytics
- Collaboration workflows
- Integration with more ML and data platformsments that we're proud of
What we learned
What's next for LineageMedic
Built With
- ai
- datahub
- docker
- fastapi
- github-actions
- graphql
- mcp
- python
- react
- typescript
Log in or sign up for Devpost to join the conversation.