ChangeProof

Evidence, not vibes, for AI-generated changes.

Comment

Inspiration

Coding agents can produce a convincing implementation in minutes, but reviewers still have to answer a harder question: did the change actually satisfy the product contract? Important gaps often live between artifacts. A ticket asks for admin-only access, the diff checks only authentication, and the happy-path test still passes.

We built ChangeProof to make those gaps visible. It treats requirements, code, and tests as one evidence set and gives reviewers a structured argument they can inspect instead of a generic AI summary.

What it does

Users paste acceptance criteria, a unified git diff, and optional test or CI output. ChangeProof returns:

  • A coverage score and merge verdict.
  • A row-by-row evidence matrix for every acceptance criterion.
  • Security, reliability, and operational risks with locations and mitigations.
  • Prioritized missing tests with executable suggestions.
  • A short next-action path and exportable Markdown/JSON report.

A built-in API-key-rotation example loads automatically. Demo mode requires no credentials and catches missing admin authorization, rate limiting, and audit attribution even though the supplied tests pass.

How we built it

ChangeProof is a dependency-free Node.js 20 application with a browser-native frontend. The server exposes a small evidence API and keeps OpenAI credentials off the client.

Live analysis uses the OpenAI Responses API with the gpt-5.6 model alias and strict JSON Schema Structured Outputs. GPT-5.6 reasons across the complete requirements, diff, and test context while the schema guarantees a stable report contract for rendering and export.

The deterministic Demo engine implements the same contract so judges can test the complete experience immediately without an API key. It is clearly labeled Demo and never claims to be a model response.

Codex accelerated the entire build: challenge research, idea selection, schema-first architecture, implementation, responsive UI, GPT-5.6 integration, tests, documentation, HTTPS deployment, GitHub publication, and submission materials. A key product decision made with Codex was to build an evidence workspace rather than another chat interface.

Challenges we ran into

The hardest design problem was separating plausible claims from actual evidence. A diff may mention a requirement keyword without implementing the behavior, while a passing test may exercise only the happy path. We solved this by classifying every criterion as covered, partial, or missing and requiring a concise evidence statement for each result.

We also needed a judging path that works without exposing credentials. Demo and Live modes share one report schema and drive the exact same interface, while the OpenAI API key remains server-side.

Accomplishments that we're proud of

  • The first screen is a complete interactive audit rather than a setup page.
  • Every conclusion is traceable to a requirement, code location, or test signal.
  • The hosted demo works without dependencies or credentials.
  • The repository includes a real GPT-5.6 Responses API path with Structured Outputs.
  • Reports export directly to Markdown or JSON for pull requests and CI.
  • Three focused tests verify the sample verdict, adaptive fallback mapping, and input validation.

What we learned

The most useful role for a frontier model in code review is not writing more prose. It is reconciling multiple artifacts, tracking missing evidence, and returning a structure humans can challenge. GPT-5.6's context handling and structured-output support make that workflow practical.

What's next

  • GitHub App integration that audits pull requests automatically.
  • Repository-aware links to exact lines and symbols.
  • CI status checks that block merges on uncovered critical criteria.
  • Team policy packs for security, privacy, and reliability.
  • Longitudinal tracking of requirements that repeatedly escape review.

Try it

The hosted demo is available at keliu.xmyingshiyun.com/changeproof. The public source branch includes setup instructions, sample data, supported platforms, an MIT license, and the full GPT-5.6 integration.

Built With

Share this project:

Updates