Inspiration
As a student, AI coding tools have changed how quickly I can build and debug things. The awkward part comes after the agent says, “fixed.” Sometimes it really is fixed. Other times the error moved somewhere else, a test was changed, or the solution only works inside the same messy workspace.
An agent reporting that its own patch works is a bit like grading its own homework. ProofPatch started from a simple question: can the fix be checked independently before I trust it?
What it does
ProofPatch sits between a bug report and an accepted patch.
It first reproduces the reported failure and records exactly what happened. The coding agent then works inside a separate Git clone, while the original repository stays untouched. ProofPatch captures the resulting patch and tests it again in a fresh environment.
A patch is accepted only when the original failure no longer appears and the configured regression checks still pass. Successful runs produce a receipt tied to the original commit, patch hash, verification environment, failure contract, and evidence history.
If the patch fails verification, ProofPatch rejects it. No optimistic “looks good to me” shortcut.
How we built it
I built ProofPatch as a Python command-line tool using Git, Docker, SQLite, structured configuration, and hash-chained evidence files.
Git is used to create independent workspaces and capture patches without giving the coding agent writable access to the original repository. Docker provides the protected execution environment with restricted mounts, permissions, networking, and resources. SQLite stores run metadata, while the evidence chain records the important events in order and makes later changes detectable.
The project supports generic command-based agents as well as Claude and Codex adapters. Those adapters can generate patches, but they cannot change the verification rules or grant themselves more access.
I used Codex with GPT-5.6 throughout development to work through the implementation phases, review security boundaries, write adversarial tests, and investigate platform-specific failures. I made the main architecture and policy decisions, especially around failing closed, workspace isolation, and what evidence must be included in a valid receipt.
Challenges we ran into
The biggest challenge was that isolation is not one setting. There were a lot of smaller boundaries that all had to agree.
The patch agent needs permission to edit its own workspace, but verification commands must not be able to rewrite the source and manufacture a passing result. Docker mounts had to reject the original repository, evidence directory, home folders, credentials, and the Docker socket. Git also needed protection from repository-controlled hooks, filters, pagers, and external diff commands.
Cross-platform support caused some memorable problems too. Windows path limits, reparse points, case-insensitive paths, file locking, and SQLite cleanup behavior all needed different handling. Windows was very good at reminding me that “works on Unix” is not a portability plan.
Designing receipts was another tricky part. The receipt needed to bind all the important evidence without including its own final hash and creating a circular calculation.
Accomplishments that we're proud of
I am most proud that ProofPatch completes the entire workflow instead of stopping at patch generation. It can investigate a failure, reproduce it independently, run an agent, capture the patch, verify it in a fresh workspace, and produce an integrity checked receipt.
The project has cross platform testing on Linux, macOS, and Windows, plus Docker end-to-end and adversarial tests. It also handles fake fixes, empty patches, agent failures, timeouts, forbidden mounts, evidence tampering, invalid state transitions, and interrupted cleanup.
Another accomplishment is that the original repository remains unchanged until the user explicitly applies a verified patch. That sounds like a small detail, but it is one of the boundaries I cared about most.
What we learned
I learned that proving a patch is not only about whether a test passed. You also need to know which commit was tested, which patch was applied, what environment ran it, what failure was expected, and whether the evidence changed afterward.
I also learned that security work often lives in fairly ordinary details. Path normalization, process cleanup, lock ownership, output limits, and command arguments are not exciting demo material, but mistakes there can break the whole trust model.
More generally, this project taught me to separate generating an answer from checking an answer. That idea applies far beyond coding agents.
What's next for ProofPatch
The next step is to run ProofPatch against more real repositories and collect feedback from developers using different languages, test frameworks, and agent providers.
I also want to make first-time setup easier, improve the examples and testing instructions, and keep checking provider CLI compatibility as those tools change. Receipt inspection could become friendlier too, especially for people who want a quick explanation without reading raw evidence events.
For now, the priority is making the core workflow predictable and boring in the best possible way. Verification software probably should not be the most surprising part of your day.
Log in or sign up for Devpost to join the conversation.