Inspiration
AI can fix code remarkably quickly. But in security engineering, a plausible patch is not the same as proof that a vulnerability is fixed.
We built ProofPatch around a practical question: When an AI agent repairs an authorization flaw, can we produce independent, reproducible evidence that the forbidden access is blocked without breaking legitimate users?
What it does
ProofPatch is a reusable security regression gate for Python ASGI applications.
Developers define explicit HTTP authorization contracts describing which requests must succeed and which must be denied. ProofPatch executes those contracts, compares observed responses against expectations, and produces machine-readable JSON and human-readable HTML evidence.
A failing contract causes a nonzero exit status, allowing GitLab CI to reject authorization regressions.
ProofPatch is intentionally focused: it verifies configured local GET contracts. It does not claim to discover every vulnerability or certify an application's complete security.
How we built it
We created a synthetic FastAPI application containing a genuine broken object-level authorization scenario: an authenticated user could access another tenant's report.
Our original GitLab CI security gate demonstrated the vulnerability, passing only 6 of 8 HTTP scenarios.
We then used GitLab Duo Developer Flow to investigate and remediate the issue. The agent identified the missing tenant ownership check, implemented a minimal authorization guard, ran verification, and created a merge request.
The corrected application passed all eight HTTP contracts. GitLab Duo Code Review also examined the implementation, while final acceptance and merging remained human decisions.
We subsequently extended ProofPatch v0.2 into a reusable, installable Python package using local Codex development assistance.
To prove reuse, we tested the installed wheel against a separate synthetic parcel-delivery API with a different authorization policy:
- RED: 7/9 contracts passed; two unauthorized reads succeeded; process exited 1.
- GREEN: 9/9 contracts passed; unauthorized reads were denied and legitimate access remained intact; process exited 0.
Both runs used the same contracts and verification engine.
The final GitLab main pipeline passed all five CI jobs and 58 tests, including all 34 original tests. It also verifies the reusable wheel integration, security evidence, packaging, linting, and static analysis.
Challenges
The hardest challenge was making the evidence trustworthy.
We needed to preserve the original failing baseline, distinguish forbidden access from legitimate reads, maintain reproducible contract and engine identities, and avoid exposing environment-backed credentials in published evidence.
We also deliberately separated responsibilities: GitLab Duo performed the original AI-assisted remediation and review, local Codex helped build the reusable v0.2 verifier, GitLab CI executed the checks, and a human approved the merges.
Accomplishments
- Demonstrated a real RED-to-GREEN authorization remediation using GitLab Duo.
- Preserved links to the original issue, agent session, merge request, and CI evidence.
- Built a reusable security contract runner with JSON and HTML reports.
- Demonstrated the same verifier against a second application.
- Verified 58 passing tests and five successful CI jobs on main.
- Published MIT-licensed source, synthetic fixtures, reproduction instructions, and evidence.
What we learned
The most valuable role of AI in security engineering is not merely generating a patch. It is helping establish a workflow where observed security behavior can be independently checked, reviewed, and reproduced.
Human acceptance remains an explicit boundary.
What's next
We would like to extend the contract model beyond configured GET requests, support additional application frameworks, and integrate richer security verification into GitLab workflows.
These are future directions—not capabilities claimed by the current prototype.
Evidence and source
Built With
- asgi
- bandit
- ci/cd
- duo
- fastapi
- gitlab
- pytest
- python
