AutoPatch

• 

  Architecture Diagram

• 

  Main Dashboard

• 

  Gitlab MR

Inspiration

Every data team has lived the 2 AM operational nightmare: an upstream column gets renamed in a source system, Fivetran syncs the change silently, and by 8 AM the CFO's production revenue dashboard is completely broken. Nobody finds out until the executive board meeting starts.

We asked ourselves: What if an AI agent could catch this automatically, trace the blast radius, and deploy a code fix before anyone even wakes up? That is why we built AutoPatch.

What it does

AutoPatch is an autonomous AI agent that acts as an automated on-call data engineer to protect data pipeline integrity end-to-end:

• Detects Schema Drift: Monitors active Fivetran connectors in real-time to intercept schema alterations immediately.
• Diagnoses Blast Radius: Traces broken downstream dbt models and automatically evaluates the downstream business impact.
• Deploys Automated Fixes: Rewrites broken SQL transformation code and opens a production-ready GitLab Merge Request.
• Generates Root-Cause Reports: Delivers a clear, plain-English incident summary detailing the issue, impact, and executed resolution in under 30 seconds.
• Visualizes Pipeline Health: Displays live agent logs, active sync states, and connector status via an interactive Streamlit dashboard.

How we built it

AutoPatch is built on the Google Agent Development Kit (ADK) using Gemini models as the core reasoning engine. The agent executes a deterministic, 5-step autonomous loop: Detect → Assess → Read → Fix → Report, where each phase is orchestrated via dynamic tool calls.

• Fivetran MCP: Plugs directly into live data pipeline feeds to capture structural changes immediately.
• BigQuery: Serves as the analytical ground truth to verify actual table schemas post-sync.
• GitLab MCP: Empowers the agent to inspect production repository files, isolate broken SQL, and programmatically open patch branches and Merge Requests.
• Arize Phoenix: Instruments every underlying agent tool call with OpenTelemetry tracing to ensure transparency and monitor multi-step execution.
• GCP Ecosystem: Runs securely on Google Cloud Platform, utilizing Secret Manager to protect all API credentials, tokens, and system keys.

Challenges we ran into

• Asynchronous ADK Session Handling: Google ADK’s session lifecycle management is strictly asynchronous. Managing execution states across multiple sequential API workflows required implementing precise event-loop handling to eliminate coroutine errors and race conditions.
• Differentiating Renames from Deletions: Fivetran handles schema changes by preserving old columns and adding new ones to prevent data loss. Building robust delta-comparison logic was essential to help the agent distinguish true structural renames from net-new additions.
• Quotas and Token Resilience: Chaining complex tool evaluations across multiple distinct platforms can exhaust standard API limits rapidly. We designed a custom backoff-and-retry structure to keep the multi-step agent loop stable under load.
• High-Fidelity SQL Parsing: Extracting accurate column dependencies directly from raw SQL files without triggering false positives or missing implicit joins took significant algorithmic refinement.

Accomplishments that we're proud of

• True Zero-Human Intervention: Built a production-ready agent capable of moving from a catastrophic schema drift alert to a verified, open GitLab Merge Request in under 30 seconds.
• Context-Aware Code Generation: The agent doesn't simply guess an isolated fix; it evaluates the exact target schema modification and updates the downstream dbt code with perfect alignment.
• Operational Visibility: Designed a clean, intuitive Streamlit dashboard that binds backend agent execution logs, BigQuery tables, and connector states into a unified view.
• Production-Grade Architecture: Followed clean development practices, including environment isolation, multi-stage building, and strict credential management via Secret Manager.

What we learned

• The Power of MCP: Model Context Protocol servers completely abstract away standard API bottlenecks. What normally takes days of parsing disparate endpoints became a highly clean, structured tool definition for the agent.
• Prompt Engineering for Multi-Step Planning: Giving an agent the freedom to run complex tasks requires highly precise prompt guardrails to guarantee execution order and prevent hallucinated tool inputs.
• Focusing on Real-World Friction: Designing an agent around a genuine, high-stakes data engineering problem creates a vastly more compelling project and practical architecture than building synthetic chatbots.

What's next for AutoPatch

• Broadening the Visual Layer: Extending past dbt models to automatically trace and patch downstream Looker views, Tableau extracts, and business intelligence dashboards.
• Triggered Execution: Migrating from manual invocations to an event-driven architecture that automatically spins up the agent immediately following a completed Fivetran sync.
• Enterprise Alerting: Adding native Slack and PagerDuty webhooks to alert teams on their phones the second an autonomous patch is opened.
• Predictive Optimization: Utilizing historical telemetry data from Arize Phoenix to trace recurring structural patterns and flag potential pipeline risks before they cause downstream failures.
• Multi-Source Expansion: Expanding the connector coverage beyond initial sources to handle Salesforce, enterprise databases, and major cloud warehouses.

Built With

• arize-phoenix
• bigquery
• fivetran
• gemini
• gitlab
• google-agent-development-kit
• google-cloud
• google-cloud-run
• google-secret-manager
• python
• streamlit
• uv