Architecture - 2
System Status
Verification Ledger
Results
Architecture - 1
Live Demo
Core Features
Creating a New Task
About
Workbench

Inspiration

AI can generate code in seconds, but generated code is not necessarily correct, secure, or reliable. A solution can look convincing, pass a basic syntax check, and still fail when it encounters edge cases or unexpected inputs.

We wanted to address a fundamental problem in autonomous software engineering: How can we trust code written by AI without blindly trusting the AI itself?

That question inspired VERIQ — Where AI-Generated Code Earns Its Proof. Instead of building another code-generation assistant, we set out to build an evidence-driven engineering agent that treats every generated patch as a hypothesis to be tested, not a result to be accepted. Our guiding principle became simple: Code is generated. Proof is earned.

What it does

VERIQ is an autonomous engineering agent that generates code, verifies its behavior, attempts bounded repairs when verification fails, and records the evidence behind its final status.

Its core capabilities include:

Intelligent model routing: Uses NVIDIA Nemotron models through Nebius Token Factory, routing tasks according to complexity to balance reasoning capability, latency, and inference cost.
Documentation-backed research: Uses Tavily to retrieve relevant documentation and technical context before implementation.
Autonomous code generation: Converts natural-language requirements into proposed code changes.
Deterministic verification: Performs Python AST-based syntax validation and executes code in a Docker sandbox with network access disabled.
Automated testing: Runs pytest to evaluate generated implementations against defined behavioral expectations.
Bounded self-repair: Uses diagnostic information, including test failures and tracebacks, to guide limited repair attempts.
Proof Ledger: Records verification outcomes as VERIFIED, SUPPORTED, FAILED, UNRESOLVED, or NOT_RUN, making the distinction between proven results and incomplete checks explicit.
Auditable evidence: Uses SHA-256 evidence fingerprints and SQLite WAL-backed storage to preserve traceable verification records.
Portable reports: Exports evidence and results in Markdown and JSON.

Our demonstration task implements population standard deviation with boundary safeguards. Rather than stopping when the code is generated, VERIQ checks the implementation and records the resulting evidence.

How we built it

We built VERIQ as a modular, full-stack application combining AI inference, research, execution, verification, and evidence management.

Our technology stack includes:

AI inference: NVIDIA Nemotron models through Nebius Token Factory.
Research: Tavily for documentation retrieval.
Backend: Python and FastAPI.
Frontend: Streamlit, with server-sent events (SSE) for streaming execution progress.
Verification: Python AST checks, pytest, and isolated Docker execution.
Evidence storage: SQLite with WAL mode and SHA-256 fingerprints.
Deployment: Docker-based execution and a deployed web demonstration.

The workflow begins with a natural-language task. VERIQ gathers relevant context, selects an appropriate model tier, generates a proposed implementation, and passes it through deterministic checks. If verification fails, the agent can use the diagnostics to attempt a bounded repair. The resulting evidence is recorded in the Proof Ledger and can be exported for inspection.

We also used Google Antigravity, Gemini (Argon model), and ChatGPT to support development, architectural exploration, implementation, and debugging. These tools accelerated our work, while VERIQ's own verification pipeline remained responsible for evaluating generated code.

Challenges we ran into

1. Balancing reasoning quality, speed, and cost

Using the largest model for every task would be inefficient. We implemented multi-tier routing so simpler edits could use faster models while more demanding tasks could be escalated when necessary.

2. Making autonomous repair genuinely useful

A model can generate a plausible fix without actually resolving the underlying failure. We addressed this by feeding relevant diagnostics into bounded repair attempts and rerunning verification instead of accepting the model's claims.

3. Executing generated code safely

AI-generated code must be treated as untrusted input. We integrated Docker-based sandbox execution with network access disabled, alongside syntax validation and automated tests, to reduce the risks of executing generated code.

4. Preserving trustworthy evidence

A simple success message does not explain what was checked or what remains uncertain. The Proof Ledger records distinct outcomes so that failed, incomplete, or unexecuted checks are not misrepresented as successful verification.

5. Integrating the complete workflow

Connecting model inference, documentation retrieval, streaming progress, code execution, testing, repair, and persistent evidence required careful coordination across the application. We focused on making these components work as one coherent engineering pipeline rather than a collection of disconnected AI features.

Accomplishments that we're proud of

Our biggest accomplishment is building a working prototype around a principle we believe is essential for AI-assisted engineering: verification must be independent of generation.

We are proud of the Proof Ledger, which makes verification outcomes explicit instead of presenting every generated solution as equally trustworthy. We also combined model routing, documentation-backed research, isolated execution, automated testing, bounded self-repair, and exportable evidence in one workflow.

In our project-reported benchmark across five coding tasks, VERIQ achieved:

100% task pass rate, compared with 60% for our single-model baseline.
54.3% lower latency, with reported latency of 1,760 ms versus 3,850 ms.
62.1% lower inference cost, with reported costs of $0.365 versus $0.962.

These results are specific to our five-task evaluation and are not intended to establish universal superiority over other models or architectures. We also achieved 48/48 passing tests in our reported project test suite.

Most importantly, we built more than a code-generation interface: we built a workflow that attempts to establish evidence for why a generated change should be trusted.

What we learned

We learned that building a useful autonomous agent involves much more than choosing a capable language model. Model selection, prompt design, context retrieval, tool orchestration, execution boundaries, failure recovery, and evaluation all matter.

We also learned that a successful generation is not the same as a successful implementation. Syntax checks can establish that code parses, while tests provide evidence about specific behaviors. Neither should be presented as proof of properties that were never checked.

Another important lesson was the value of explicit uncertainty. A trustworthy system should distinguish what passed, what failed, what remains unresolved, and what was never tested. That distinction became central to our Proof Ledger.

Finally, model routing showed us how system-level design can improve efficiency. Instead of depending on a single model for every task, an agent can allocate reasoning effort according to complexity and use independent verification to decide whether additional work is necessary.

What's next for VERIQ: Where AI-Generated Code Earns Its Proof

We see VERIQ evolving into a more comprehensive verification-first engineering platform.

Our next priorities include:

Broader language support: Extend verification beyond Python to additional programming languages and development ecosystems.
Richer test generation: Generate targeted tests for edge cases, boundary conditions, regressions, and failure scenarios.
Stronger security analysis: Add static security analysis, dependency checks, and more granular sandbox restrictions.
Improved evidence visualization: Make it easier to inspect execution traces, failed assertions, repair history, and evidence fingerprints.
Expanded benchmarking: Evaluate more tasks and compare different model-routing strategies under repeatable conditions.
Developer workflow integration: Explore pull-request reviews and CI/CD integrations so verification evidence can accompany proposed code changes.
More rigorous verification: Expand the distinction between passing tests and stronger forms of assurance, including property-based testing and other techniques where appropriate.

Our long-term vision is an engineering agent where every proposed change comes with a transparent record of what was checked, what passed, and what remains uncertain.

We do not believe AI-generated code should be trusted simply because a model produced it. We believe trust should be built through observable, reproducible evidence.

VERIQ — Code is generated. Proof is earned.