PatchGuard: An AI Agent That Guards Your Code Against Regressions

Inspiration 💡

Every developer knows the frustration of fixing one bug only to introduce another. A seemingly small code change can break a previously working feature, and identifying the exact change responsible can take hours of debugging.

We built PatchGuard to address this problem: what if an AI coding agent could not only help modify code, but also verify that those modifications didn't break anything?

Our vision is to make coding agents more reliable by giving them a safety net. Instead of blindly trusting generated code, PatchGuard tracks code changes, runs tests, diagnoses failures, and helps developers return to a known-good version when a regression occurs.

What It Does 🛡️

PatchGuard is an AI-powered coding regression detection and recovery system designed to make code changes safer.

Its core workflow includes:

• Version Tracking: Maintains successive versions of code patches so changes can be examined and compared.
• Diff Analysis: Uses unified diffs to identify what changed between versions.
• AST-Based Analysis: Analyzes Python source code using Abstract Syntax Trees (ASTs) to track functions and understand structural changes beyond simple text differences.
• Automated Testing: Executes tests using pytest to determine whether a code version behaves as expected.
• Failure Diagnosis: Uses AI-assisted reasoning to help explain failures and identify potentially problematic changes.
• Recovery: Tracks the latest known passing version, providing a foundation for restoring a working state after a regression.

Our goal is simple: make AI-generated code easier to trust by verifying its behavior instead of assuming it is correct.

How We Built It ⚙️

We designed PatchGuard around a hybrid architecture that combines deterministic software engineering tools with AI-assisted reasoning.

1. Code Versioning and Diff Analysis

We implemented patch version tracking and used Python's difflib to generate unified diffs between versions. This provides a clear view of how the code evolved and which modifications might have introduced a problem.

2. Structural Code Analysis

We used Python's built-in ast module to inspect source code and track function-level changes. This allows PatchGuard to reason about modifications to individual functions rather than relying exclusively on line-by-line comparisons.

3. Test-Driven Regression Detection

We integrated automated testing with pytest. By recording test outcomes for successive code versions, PatchGuard can distinguish a version that passes validation from one that introduces a failure.

4. Checkpoint and Recovery Mechanism

We implemented checkpoint tracking to record test results and identify the most recent passing version. The system uses version sequence information to maintain a consistent view of code history.

5. AI-Assisted Diagnosis

We designed the diagnosis component to complement deterministic checks. While the testing and diff systems establish what changed and whether tests failed, the AI component helps interpret the evidence and explain potential causes.

This separation is intentional: the AI reasons about the problem, while deterministic tools provide verifiable evidence.

Challenges We Faced 🧩

Combining AI Reasoning With Deterministic Validation

An AI model can generate convincing explanations that are not necessarily correct. We addressed this architectural challenge by separating AI-assisted diagnosis from objective checks such as code diffs and test execution.

Identifying the Source of a Regression

Knowing that a test failed is not the same as knowing why it failed. We incorporated diff analysis and AST-based function tracking to help narrow the investigation to relevant code changes.

Maintaining Reliable Checkpoints

A recovery mechanism must not mistake a newer version for a better version simply because it was created later. We introduced explicit test-result tracking and version sequencing to identify the latest known passing checkpoint.

Designing a Reliable Agent Workflow

We learned that a useful coding agent needs more than an LLM and a prompt. It needs structured state, tools, validation, and clear rules for deciding what to do next.

What We Learned 📚

Building PatchGuard helped us understand several important concepts in AI engineering and reliable software development:

• How to use ASTs for structural source-code analysis.
• How unified diffs can help explain the evolution of a codebase.
• How to integrate automated testing into an agentic workflow.
• Why version history and checkpoint management are essential for safe recovery.
• How to combine LLM-based reasoning with deterministic validation.
• Why an AI agent should be evaluated on verified outcomes, not just plausible explanations.

Most importantly, we learned that building a reliable AI agent is not just about making a model smarter; it is about designing a system that can verify its actions and recover when things go wrong.

What's Next? 🚀

PatchGuard is a foundation for a more autonomous coding safety agent. Our next steps include expanding its test-generation capabilities, strengthening regression localization, integrating diagnosis with an AI coding workflow, and developing a more complete recovery process.

We envision a future where developers can experiment with AI-generated code more confidently because every change is accompanied by evidence that it works—and a clear path back when it doesn't.

PatchGuard: Write code boldly. Verify every change. Recover with confidence.

Built With

• api
• nvidia
• python