Inspiration
I had 200+ Dependabot alerts on a personal repo. Every few weeks I'd open the security tab, feel guilty, close it. The alerts kept growing. Dependabot opens PRs, but it doesn't tell you whether a major bump will break your code — and nobody has time to read every changelog, grep for renamed APIs, and verify CI. I wanted an agent that would do the full triage on its own: figure out what's safe, open the PRs, and fix what breaks.
How I built it
Built solo with Claude as a coding partner.
Skeleton. Got the four pipeline stages wired end-to-end: Prioritiser, Safety Analyser, Executor, Monitor. Cloud Run deploy and a basic dashboard went up the same night. The pipeline could read alerts and produce verdicts, but the analysis was shallow — string matching on release notes.
Real analysis. Added structured findings with file and line citations, posted as inline PR review comments. Supplemented release notes with CHANGELOG.md for major bumps. Added a compile check and type-diff tool so the Safety Analyser could verify breakage against real TypeScript types instead of guessing from changelogs.
Reliability. Refactored code search to chunk queries and filter to source files. Added campaign resume so interrupted runs pick up where they left off. Fixed multi-file detection bugs where the scanner was missing affected files.
Agents and self-healing. Replaced the old safety classifier (a single LLM call) with a Genkit agent that has tools — file reading, compile check, type diff, package docs. Added the Coder Agent to write fixes for broken upgrades. Switched to webhook-driven CI monitoring. Added real-time dashboard updates. Then spent a long stretch fighting state corruption: concurrent webhooks overwriting each other's verdicts, fix attempts getting stuck, the agent editing package-lock.json and breaking installs. Each fix surfaced a new edge case.
The fix cycle. The autonomous loop — CI fails → re-analyse with error logs → Coder Agent writes a patch → poll CI → repeat — took the longest to get right. First version relied on webhooks to trigger re-analysis after each fix, but GitHub's check_suite events arrived with empty data when commits were pushed seconds apart. Switched to active CI polling. Then discovered stale Firestore reads were causing the re-analysis to repeat the same verdict, so each iteration now reads fresh state. Final fix: made the whole loop self-contained and required a terminal verdict before marking a campaign done.
Challenges
State races. The fix cycle has a lot of moving parts: a bump goes from risky to reanalysing to fixing to pending and back. Multiple webhooks can fire at once for the same bump. Without coordination they overwrite each other's state. I ended up with a lease pattern in Firestore — before touching a bump, the handler claims it with a timestamp. If another webhook finds the lease held, it backs off. Leases expire after 10 minutes so a crashed handler doesn't block everything permanently.
Silent CPU kills. The webhook handler sends 200 right away and does the heavy work in the background. Cloud Run cuts CPU after the response goes out by default. Agent responses were getting chopped mid-generation with no error. Took a while to figure out what was happening — the fix was a single deploy flag (--no-cpu-throttling).
Teaching the Coder Agent to learn from failure. First version re-applied the same broken fix on retries. Three things made it work: feeding CI error logs as the source of truth (not the changelog), showing what previous attempts tried, and making it read the current file before editing. A later change gave it its own compile check so it could verify a fix before committing.
Knowing when a fix worked. The Coder Agent's natural-language response was unreliable for figuring out whether it actually fixed anything — sometimes it would say "done" when no commit was made. Switched to checking the tool call history directly: if commitFix returned success: true, the fix landed, regardless of what the agent said about it.
What I learned
- Webhooks aren't reliable event delivery when state changes fast. GitHub's
check_suiteevents drop data when commits land seconds apart. You need a polling fallback. - Leases beat locks for coordinating concurrent webhook handlers in Firestore. Simpler to implement, and the timeout gives you crash recovery for free.
- Cloning repos is expensive in a serverless container. The GitHub Git Data API lets you read and write files, create commits, and update branches through HTTP calls — no clone, no filesystem, no memory spike.
- Check what the agent did, not what it said. Tool call history is ground truth. The model's summary of its own work isn't.
- Cloud Run kills background work silently. If your handler does anything after responding, you need
--no-cpu-throttling.
Built With
- firestore
- gcp
- genkit
- node.js
- typescript
Log in or sign up for Devpost to join the conversation.