PatchTrace

Inspiration

With AI tools now capable of generating code fixes in seconds, developers receive security alerts and instant patches faster than ever. However, engineering teams face a major blind spot:

bold Does the AI-generated fix actually stop the attack?

bold Did the fix accidentally break normal functionality for real users?

Too often, teams trust an AI fix, ship it to production, and later discover that the vulnerability was untouched, or worse, that the fix broke core features. We built PatchTrace to bridge this trust gap by creating an automated verification layer that proves a fix works before it goes live.

bold_ italics _ "AI writes the patch. PatchTrace traces the proof."

What it does and How we built it

PatchTrace takes a security vulnerability, asks an AI (or uses a fallback) to propose a patch, and empirically proves its safety through automated test suites.

Technical Stack & Architecture Backend Server: Built with Python, FastAPI, and Uvicorn (running on port 8000), orchestrating proof execution via native Python subprocesses.

Target Application: A Flask e-commerce store (running on port 5001) backed by an in-memory SQLite database, containing a real-world Broken Access Control bug (where logged-in user Alice can read Bob's private order).

Testing & Proof Engine: Powered by pytest to run automated security attack tests alongside functional regression checks.

AI & Diff Engine: Integrates the Anthropic Claude API (supported by a local fallback safety net), difflib + diff2html for side-by-side code diffs, and Jinja2 for downloadable verification reports.

Frontend Dashboard: Styled with Tailwind CSS, featuring real-time step tracking and state persistence saved to disk.

The Verification Workflow Reproduce: Runs pytest attack scripts against the vulnerable app to confirm the security flaw exists.

Generate Fix: Prompts Anthropic Claude for a corrected app file.

Verify Bad vs. Good Patches: If a bad patch fixes the leak by blocking all users indiscriminately, PatchTrace catches the regression failure, marks it NOT VERIFIED, and keeps deployment locked.

Deploy: Once a patch passes all security and regression tests with a VERIFIED verdict, PatchTrace allows one-click deployment to the live server.

Challenges we ran into

State Management & Process Execution: Managing two separate running servers (FastAPI backend and Flask target app) while dynamically restarting the target app with patched code via Python subprocesses required careful process handling.

Preventing Misleading Verifications: Designing an engine smart enough to detect "fake fixes" - such as patches that break the code syntax or block access to legitimate users, to ensure deployment stays locked until a fix is genuinely valid.

Offline Resilience: Ensuring the demo runs reliably without an API key or internet access by building a robust local safety net (fallback/app_fixed.py).

Accomplishments that we're proud of and What we learned

Trust Needs Proof: AI is fantastic at writing code quickly, but empirical testing (reproducing attacks and running regression suites) is essential before deploying AI output to production.

Process Automation: Building seamless integration between process execution, test runners (pytest), and visual diff rendering in a single web dashboard taught us how to construct effective DevSecOps tools.

What's next for PatchTrace

Business Model & Vision (Startup Track)Target Customer: DevSecOps leads and Engineering Managers seeking to safely adopt AI-assisted development. Business Model: A B2B SaaS model offering monthly recurring revenue (MRR) through per-developer seats or per-repository subscriptions: Starter: $49/month per repository for small teams.Enterprise: Tiered pricing with dedicated sandbox execution and compliance audit logs.Competitive Advantage: Security tools find vulnerabilities, and AI tools write fixes. PatchTrace provides the proof layer in the middle, verifying safety empirically rather than relying on trust.

Next Steps: Expanding beyond single bug types into full GitHub Action / Pull Request integrations and executing patches inside isolated Docker sandboxes.