Verity: Proof-Carrying Release Gates

Verity turns AI-assisted release promises into reproducible proof—linking every critical claim to tests, source checks, and explicit human review

Comment

Inspiration

AI coding agents can create a convincing, multi-file pull request in minutes. The difficult question is not only “do the tests pass?” but “which real release promise do these tests prove?” A green suite and an agent-written “done” message force a reviewer to reconstruct that link from scratch.

What it does

Verity is a proof-carrying release gate for AI-assisted code changes. A team writes important release claims—such as “order creation requires an authenticated customer”—then attaches reproducible evidence to each claim.

Verity can evaluate source-pattern checks, executable test commands, and explicit manual review. It generates a standalone HTML report and portable JSON evidence record. Each evidence result is ordered and SHA-256 hash-chained, so a reviewer can detect a changed, removed, or reordered item with verity verify.

The key distinction is simple: a test is not a claim. It becomes evidence only when it is explicitly connected to the customer, security, or product promise it supports. If a decision requires a human, Verity shows REVIEW instead of silently turning it green.

How we built it

We built Verity in Codex using GPT-5.6 during OpenAI Build Week. Codex accelerated the complete engineering loop: translating the judging rubric into a product thesis, designing the configuration contract, implementing the evaluator and CLI, creating the standalone report UI, building the sample API and tests, and verifying the one-command judge path.

The project is intentionally local-first: Node.js 20+, no dependencies, no API key, no account, and no network connection are needed to run the demo. GitHub Actions runs the same validation and publishes a live static demo through GitHub Pages.

Challenges we ran into

The hard product decision was resisting a false “AI-approved” result. Static checks and tests are valuable, but they cannot infer whether a change actually matches a release owner’s intent. Verity therefore treats human sign-off as a first-class REVIEW state, not an automated pass.

Accomplishments that we're proud of

  • A runnable CLI with static, executable, and manual evidence types.
  • A transparent failure path: remove the authentication guard in the sample API and the critical release claim blocks immediately.
  • Portable HTML and JSON artifacts that are easy for a reviewer to inspect.
  • Evidence integrity verification that detects changed, removed, or reordered proof.
  • A zero-install judge path: npm test, then npm run demo.

What we learned

Agentic engineering changes the bottleneck from writing code to establishing trustworthy evidence. Product-level traceability—promise to proof—is as important as generating a correct-looking diff.

What's next for Verity

We will add first-class integrations for pull requests and CI annotations, richer evidence adapters for API and browser tests, policy templates for common release risks, and signed/attested evidence artifacts for team handoff.

Built With

Share this project:

Updates