ClaimTrace

Evidence before approval: ClaimTrace links AI-generated claims to literal source quotes, binds human approval to one exact version, and blocks export when the source or review changes.

Comment

Inspiration

AI answers are easy to generate. Trustworthy approval is much harder.

A reviewer may receive a polished AI summary without knowing which exact parts of the source support its claims. Even worse, the source or the reviewed answer may change after approval while the old approval still appears valid.

ClaimTrace was built around a simple idea:

An AI-generated claim should not be approved unless its evidence is visible, and an approval should never survive an unreviewed change.

What it does

ClaimTrace is a human-in-the-loop approval system for AI-assisted work.

It uses GPT-5.6 to transform source material into:

  • a concise review summary,
  • individual structured claims,
  • and literal source quotations supporting each claim.

The reviewer can inspect and edit the proposal before making a decision.

Approval is then bound to the exact combination of:

  • the source document,
  • the reviewed AI output,
  • and that specific version.

Only an approved version can be exported.

If the source or reviewed content changes, ClaimTrace immediately:

  • expires the previous approval,
  • blocks export,
  • and requires fresh analysis and human review.

ClaimTrace performs no external action automatically. AI proposes; a person decides.

Why it is different

ClaimTrace is not just a chatbot that displays citations.

Its core is the control system around the model:

  1. Every claim must point to literal evidence.
  2. Human approval applies only to one exact version.
  3. Any unreviewed change invalidates that approval.
  4. Export remains blocked until the new version is reviewed again.

The result is a clear chain:

source → AI proposal → literal evidence → human decision → controlled export

How I built it

The interface was built with React, TypeScript, and Vite.

A Node.js and Express backend connects securely to the OpenAI Responses API using GPT-5.6. The API key remains on the server and is never exposed to the browser.

The system also includes:

  • structured JSON generation,
  • schema validation with AJV,
  • additional literal-evidence validation,
  • SHA-256 fingerprints for exact-version approval,
  • versioned workflow states,
  • guarded JSON export,
  • request and concurrency limits,
  • input limits and timeouts,
  • controlled error handling,
  • and protected hosted access.

I used Codex throughout the build to inspect the repository, implement focused changes, run tests, audit the result, and prepare the hosted evaluation build. Git and GitHub provide the change history, while Render hosts the protected demo.

Challenges

The hardest part was not generating useful text. It was designing the rules around that text.

The main questions were:

  • How can the system prove that a claim came from the source?
  • What exactly is a human approving?
  • How should approval behave when the source changes?
  • How can an old decision be prevented from silently applying to a new version?
  • How can a real hosted model be used without exposing secrets or allowing uncontrolled usage?

The solution was to treat approval as a property of one exact version, not as a permanent label attached to a document.

Accomplishments

The protected hosted build has passed a complete end-to-end test with the real model:

  1. A source document was analyzed by gpt-5.6-sol.
  2. ClaimTrace generated a summary, structured claims, and literal evidence.
  3. A human approved the exact reviewed version.
  4. The approved JSON package was exported.
  5. The source was edited.
  6. The previous approval expired immediately.
  7. Approval and export became blocked.
  8. Reanalysis created a new version requiring fresh human review.

The project also passes:

  • 44 automated tests across 8 test files,
  • TypeScript validation,
  • the production build,
  • and repository checks for secrets and unwanted artifacts.

What I learned

This project changed how I think about trustworthy AI systems.

Model quality matters, but it is only one layer. The surrounding product must also make visible:

  • what the model is claiming,
  • which evidence supports it,
  • what a human actually approved,
  • and whether that approval is still valid.

I also learned how the complete product pipeline fits together: local development, Codex-assisted implementation, testing, Git history, GitHub, protected hosting, server-side API access, and browser-based evaluation.

What's next

ClaimTrace could be adapted to workflows such as:

  • policy and compliance review,
  • operational instructions,
  • research synthesis,
  • internal decision support,
  • and document-based approval processes.

Future versions could add document uploads, team accounts, reviewer roles, persistent audit history, version comparison, and integrations with existing workflow tools.

The core rule would remain the same:

No claim without evidence. No approval without an exact version. No export after an unreviewed change.

Built With

Share this project:

Updates