ProofStack

• 

  After

• 

  Before

Inspiration

AI coding tools can generate code quickly, but a patch that looks correct is not necessarily a patch that works. Developers still need to reproduce bugs, run tests, inspect changes, and ensure existing functionality is not broken. We built ProofStack to make AI-generated code more reliable by focusing on evidence, not just suggestions.

What It Does

ProofStack is an AI coding agent that analyzes Python bugs, proposes fixes, applies patches in an isolated environment, and runs tests to verify the results. It creates a Verification Ledger containing the code changes, test outcomes, execution details, and remaining uncertainties. Instead of blindly trusting an AI-generated patch, developers can inspect the evidence behind each result.

How We Built It

We designed ProofStack around NVIDIA Nemotron models served through Nebius Token Factory. A Python backend orchestrates the agent's workflow: understanding a bug report, inspecting relevant source code, generating a repair plan, applying a patch, and executing tests. A web dashboard presents the process through a live timeline, code diffs, test results, and a final verification report.

We focused on a narrow, reproducible Python workflow so we could prioritize reliability, transparency, and a usable developer experience over unnecessary complexity.

Challenges We Faced

Building a reliable coding agent requires more than generating a plausible solution. The agent must interpret test failures, avoid unnecessary code changes, handle unsuccessful repairs, and distinguish verified results from assumptions. Safely executing generated code is another important challenge, which is why isolated execution and strict tool permissions are central to our design.

What We Learned

This project reinforced that useful AI agents need structured workflows, observable actions, bounded retries, and measurable outcomes. Test results provide valuable evidence, but passing tests do not guarantee that software is completely correct. ProofStack makes that distinction visible and helps developers make better-informed decisions.

What's Next

We aim to expand language support, improve regression detection, evaluate performance across reproducible bug-fixing tasks, and integrate with developer workflows such as GitHub pull requests. Our goal is to make AI-assisted software development more transparent, testable, and trustworthy.

Built With

• docker
• fastapi
• git
• nebius-token-factory
• next.js
• nvidia-nemotron
• pytest
• python
• react
• tailwind-css