Inspiration
What it does
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned# ChangePilot
Autonomous Data Change Engineering
Modern data teams rarely break systems because they intended to. They break them because a seemingly harmless change in one dataset can silently affect pipelines, dashboards, ML features, data contracts, and downstream teams.
A developer renames one field.
A pipeline stops working.
A dashboard starts showing incorrect results.
An ML feature becomes stale.
A data contract is violated.
The real problem isn't making the change. It's understanding everything the change could affect before it happens.
💡 The Inspiration
We wanted to build something around a simple question:
"What happens if I change this data?"
Traditional schema-change workflows often require engineers to manually search through documentation, lineage graphs, pipelines, dashboards, and ownership information before they can confidently answer that question.
We imagined an engineering agent that could understand the organization's data ecosystem and answer it automatically.
That idea became ChangePilot.
ChangePilot is an autonomous data change engineering platform that uses DataHub's metadata and lineage context to analyze proposed schema changes, determine their blast radius, calculate risk, explain why the change is dangerous, and generate an actionable remediation plan.
Instead of discovering a broken system after deployment, engineers can discover the impact before the change reaches production.
🚀 What ChangePilot Does
A user submits a proposed data change such as:
customer.email → customer.contact_email
## What's next for Change Pilot
ChangePilot analyzes the change against the available data context.
It then:
Identifies affected assets
Datasets
Pipelines
Dashboards
ML features
Data contracts
Traverses lineage
to understand how the change propagates through downstream systems.
Calculates a deterministic risk score
based on factors such as downstream asset count, production pipeline impact, criticality, and governance dependencies.
Explains the risk
in human-readable language rather than simply displaying a number.
Generates an engineering remediation plan
with prioritized actions for the teams responsible for affected systems.
The result is a workflow that transforms:
"I want to rename a field."
into:
"Here is everything this change affects, here is why it is risky, and here is how to execute it safely."
🧠 Why DataHub Matters
DataHub is fundamental to the ChangePilot concept.
The power of the system comes from treating metadata and lineage as engineering context, rather than simply as documentation.
A schema change is only meaningful when we understand its relationships to the rest of the organization.
For example:
customer
↓
customer_360
↓
marketing_customers
↓
customer_growth_dashboard
But the impact may also propagate into:
customer
├── email_marketing_pipeline
├── customer_data_contract
├── customer_contactability
└── churn_prediction_features
This context allows ChangePilot to reason about the change as a dependency graph instead of an isolated database operation.
🤖 From Analysis to Action
We didn't want to build another lineage viewer.
We wanted to build an engineering agent on top of metadata.
ChangePilot therefore goes beyond:
"These assets are affected."
It attempts to answer:
"Why are they affected?"
and then:
"What should the engineering team do about it?"
For a high-risk change, ChangePilot can produce a sequence of remediation actions such as:
establish backward compatibility
update data contracts
migrate production pipeline references
update ML feature definitions
update downstream analytics and dashboards
The goal is to move data engineering from reactive debugging toward proactive change management.
🏗️ How We Built It
We built ChangePilot as a web-based engineering platform with a workflow centered around proposed data changes.
The application includes:
Change Analysis — submit and evaluate proposed schema changes
Lineage Graph — visualize downstream dependencies
Risk Assessment — calculate and explain change risk
Remediation Engine — generate prioritized engineering actions
DataHub Explorer — inspect datasets, schemas, ownership, and governance context
Demo Scenarios — reproduce realistic enterprise data-change situations
The interface was designed to feel like an internal platform that a real data engineering organization could use, rather than a simple hackathon dashboard.
📊 Our Demo Scenario
Our primary demonstration starts with:
customer.email → customer.contact_email
At first glance, this looks like a simple rename.
ChangePilot discovers that the field participates in multiple downstream dependencies.
The system identifies:
multiple downstream datasets
a production pipeline
a data contract
an ML feature
compliance-related data
a business intelligence dashboard
The change receives a 95/100 critical risk score in our demonstration scenario.
More importantly, ChangePilot explains why the score is high and provides a five-step remediation strategy.
This is the core experience we wanted to demonstrate:
A tiny schema change can have an organization-wide impact. ChangePilot makes that impact visible before it becomes an incident.
📚 What We Learned
The biggest lesson was that metadata becomes significantly more powerful when it is treated as context for intelligent systems.
Lineage isn't just something engineers look at after something breaks.
It can become part of an agent's reasoning process.
We also learned that building an effective engineering agent requires more than generating an answer. The system needs to provide:
context
evidence
explainability
prioritization
actionable next steps
That shaped the design of ChangePilot.
⚡ Challenges
One of our biggest challenges was translating a complex data dependency problem into an experience that engineers could understand immediately.
We had to think carefully about how to represent:
multi-hop lineage
different asset types
ownership
governance relationships
risk factors
remediation dependencies
Another challenge was separating authoritative metadata context from the conclusions generated by the analysis layer.
This led us to make the lineage, risk calculation, explanation, and remediation workflow explicit in the product instead of hiding everything behind a single AI response.
🔮 What's Next
ChangePilot's ultimate goal is to become a continuous Data Change Control Plane.
With deeper production integration, the system could:
Detect changes automatically.
Analyze their blast radius before deployment.
Identify affected owners and teams.
Generate migration plans.
Create or update engineering tasks.
Validate changes against data contracts.
Monitor the migration.
Record the resulting decisions and context back into the data ecosystem.
The long-term vision is simple:
Before your data changes break production, let an agent understand the consequences and help you change it safely.
🏆 Why ChangePilot?
Data infrastructure is becoming increasingly complex.
Organizations don't just have databases anymore. They have pipelines, warehouses, ML systems, dashboards, contracts, governance policies, and dozens of teams depending on the same data.
Humans shouldn't have to manually reconstruct that dependency graph every time someone proposes a schema change.
ChangePilot turns organizational data context into engineering intelligence.
Instead of asking:
"What broke?"
we want teams to ask:
"What will break if we do this?"
And get the answer before production does.
## **Built With**
For the Devpost tags, I recommend keeping them tightly relevant rather than filling all 25 slots:
**DataHub, Data Engineering, Data Lineage, Metadata, AI Agents, Artificial Intelligence, Generative AI, Data Governance, Schema Evolution, React, TypeScript, JavaScript, Google AI Studio**
### One thing I would change before final submission
Your current UI is already strong visually. The **biggest thing that can elevate ChangePilot from a polished prototype to a serious hackathon contender is making the DataHub connection genuinely functional**, especially the path:
**DataHub → metadata/lineage → ChangePilot agent → analysis → remediation → DataHub**
That will make the story above match the actual implementation rather than merely describe an ambitious future.
Log in or sign up for Devpost to join the conversation.