PatchFlow

Ship APIs that don't break in production. PatchFlow autonomously finds every failure gap, reads your code, and delivers the fix as a pull request before your users find it first.

Comment

Inspiration

Every developer has shipped code that worked perfectly in development and silently failed in production. Not because the logic was wrong but because nobody tested what happens when the database drops a connection mid-request, when a third-party API returns a 429 at checkout, or when an upstream service responds with malformed JSON instead of the expected shape.

These are not edge cases. They are guaranteed to happen. The question is whether your API handles them gracefully or leaks a raw stack trace to whoever is on the other end.

Chaos engineering deliberately breaking your system to find its weaknesses has existed for years. Netflix pioneered it. Tools like Gremlin and k6 formalized it. But every existing solution stops at the same point: "here is what broke."

Someone still has to read the logs, locate the right file, understand the failure context, write the fix, review it for correctness, and open a pull request. For teams with dedicated SREs, that process takes hours. For solo developers and small teams, it often just doesn't happen at all.

I built PatchFlow because I wanted to close that gap entirely not just find what breaks, but deliver the fix.

What it does

PatchFlow is a six-agent autonomous pipeline that takes your API from vulnerability discovery to a reviewed, ready-to-merge pull request without a human touching anything in between.

1. Discovery Agent Parses an OpenAPI spec, Postman Collection, or manual endpoint list into a complete grouped inventory. High-risk endpoints (admin, destructive, webhook) are flagged and unchecked by default. Nothing runs without deliberate user opt-in.

2. Chaos Agent Injects 18 real failure modes against each selected endpoint via actual HTTP requests not mocks. Timeouts, database connection drops, malformed JSON bodies, rate limit responses, empty bodies, wrong content types, and more. Records exactly how the app responded: status code, whether internal error details leaked to the caller, response time.

3. Analyst Agent Looks across all results and identifies patterns. Not just "this endpoint failed" — but "every database failure across every endpoint leaks a raw SQLAlchemy traceback to the client." Calculates a risk score from 0–100 and produces findings ranked by severity: CRITICAL, HIGH, MEDIUM, LOW.

4. Repo Context Agent Clones the user's GitHub repository and reads the actual source code. For each finding, it follows the call chain across files route declaration → service layer → database repository — up to three levels deep. Captures the real function body, existing imports, and error-handling conventions already used in that codebase. Most AI code tools generate fixes against an imagined version of your code. PatchFlow reads the real file first.

5. Fix Agent Uses the real code context to write an actual patch same variable names, same style, same import conventions already in the file. When a pattern repeats across many endpoints, it generates a single middleware fix rather than patching each route individually.

6. Review Agent Before any pull request is opened, the Review Agent independently audits every fix. It checks correctness, missing or duplicate imports, style consistency with the surrounding codebase, and potential regressions. If it finds problems, it rejects the fix and sends structured feedback back to the Fix Agent what was wrong and why. The Fix Agent revises and resubmits. This loop continues until the fix is approved.

Only after Review Agent approval does the GitHub Agent open a pull request one per critical/high finding, on a dedicated branch, using the user's own OAuth token so every PR appears under their own GitHub account.

Every agent step, every thought, every tool call, every rejection and revision streams to the frontend in real time via WebSocket so the user watches the full reasoning process live.

How we built it

AI Layer — Qwen Cloud Every agent uses qwen3.7-plus via Qwen Cloud's OpenAI-compatible endpoint. Each agent runs a ReAct (Reason + Act) loop — Qwen reasons about what to do, calls a tool, observes the result, and iterates until it reaches a conclusion. The Fix → Review → Fix correction loop is the most sophisticated usage: the Review Agent's rejection message becomes structured input to the Fix Agent's next iteration, creating a genuine self-correcting cycle powered entirely by Qwen reasoning.

Backend — FastAPI + PostgreSQL Six database tables track sessions, endpoints, failure results, agent steps, reports, and pull requests. Every agent thought, tool call, and observation is persisted as an AgentStep row and pushed to the frontend via WebSocket in real time. The full agent pipeline runs as a FastAPI background task so the HTTP response returns immediately and progress streams live.

GitHub Integration Users authenticate via GitHub OAuth. The GitHub Agent uses each user's own OAuth token not a static service account to open PRs on their repositories. Every PR is opened by the actual developer. A fallback static token covers local development only.

Frontend — Next.js + TypeScript Five pages: landing, dashboard with aggregate analytics, new session wizard with four-tab API source input, live session page with dual-panel layout (agent reasoning trace left, failure result grid right), and a report page with side-by-side code diffs and pull request status tracking.

Infrastructure — Alibaba Cloud ECS + OSS Backend deployed on Alibaba Cloud ECS (Ubuntu 22.04, Nginx reverse proxy, systemd process management). Session reports exported to Alibaba Cloud OSS with publicly shareable links generated via the OSS SDK.

Challenges we ran into

Multi-file code traversal in the Repo Context Agent The naive approach — find the route file, patch it — breaks immediately on any real codebase. Route handlers are thin wrappers. The actual logic lives in a service file. The database call lives in a repository file. The bug causing an unhandled exception might be three files away from the route declaration. Teaching the agent to follow import chains, decide when it had found the actual problem location versus just a delegation point, and capture enough context for a stylistically consistent fix required significant iteration on both tool design and prompting strategy.

Building a genuine Fix → Review → Fix correction loop The straightforward version of a review agent just flags problems. Making it reject a fix and send structured feedback back to the Fix Agent — and having the Fix Agent actually incorporate that feedback precisely rather than regenerating the same output — required careful design of the inter-agent message format. The feedback had to be specific enough to act on, not just generic criticism.

Endpoint discovery without guessing Early versions tried to auto-detect endpoints by probing common paths or scanning codebases. Too many false negatives on real apps, too many tokens wasted on large codebases. The solution was to stop guessing and adopt the same inputs developers already use: OpenAPI specs, Postman Collections, or manual entry.

Keeping autonomous actions safe An agent injecting failures into a live API could cause real damage if it targeted delete endpoints or payment flows without consent. The endpoint selection screen solves this — every endpoint shown grouped by tag, destructive operations flagged and unchecked by default. Nothing runs without deliberate opt-in.

Accomplishments that we're proud of

The self-correcting Fix → Review loop produces genuinely better code than a single-pass generation. The first fix the Fix Agent produces is rarely the best one. The Review Agent catching a missing import or a duplicate exception handler and sending it back for revision is exactly how real code review works — and the output quality reflects it.

The Repo Context Agent's multi-file traversal works on real codebases. Following call chains across route → service → repository files and identifying the correct insertion point for a fix — not just the closest file to the route — is a non-trivial reasoning task that the agent handles reliably.

Every agent step is observable in real time. The live reasoning trace isn't a marketing feature — it's architecturally fundamental. Every thought, tool call, rejection, and revision is persisted and streamed. The system is transparent by design, not by accident.

The endpoint selection UX mirrors tools developers already trust. Postman, Insomnia, and Bruno all show you a complete inventory before running anything. That pattern exists for good reason, and PatchFlow follows it.

What we learned

Building a multi-agent pipeline taught me that the hardest problem isn't making individual agents capable it's making them work together reliably. Each agent produces structured output the next agent depends on. When output is ambiguous, the failure propagates silently and surfaces far from its actual cause. Making every agent's output schema explicit and validated was the single most impactful architectural decision.

The Fix → Review loop specifically taught me that rejection is a feature, not a failure. The same principle that makes human code review valuable first drafts have problems, structured feedback improves them applies directly to agent pipelines. The loop exists because it produces better output, not as a safety theatre.

I also learned that the most useful feature of an autonomous system is often knowing when not to act autonomously. The endpoint selection screen, the Fix → Review loop, the PR review before merge these checkpoints are not limitations. They are what makes the system trustworthy enough to use on a real codebase.

What's next for PatchFlow

  • Scheduled reliability scans — run automatically on every deployment via CI/CD integration, not just on demand
  • Team-level reporting — aggregate risk scores across multiple services for engineering leads
  • Broader language support — Fix and Review agents are currently strongest on Python/FastAPI, with Express/Node.js as the next target
  • Review Agent metrics — tracking fix rejection rates per finding type to identify which failure modes consistently produce the hardest-to-fix gaps
  • Review Agent expansion — currently checks correctness, imports, duplicates, style, and regressions; next additions are security-specific checks (injection patterns, sensitive data exposure in error messages) and test generation alongside the fix

Built With

Share this project:

Updates