Why I built PolicyProof I built PolicyProof as a solo project after seeing how easily a review fragments across a written policy, a purchase order, an invoice, approval evidence, and reviewer comments. The hard part is not only finding information. It is showing exactly why a conclusion was reached, reproducing it, and keeping the final decision accountable to a person. The product is built around one chain: Policy → Control → Evidence → Conclusion → Human decision
What it does PolicyProof turns a written procurement policy into reviewable controls, links every conclusion to exact evidence, runs objective checks deterministically, and records the final human decision. The Northstar demo contains seven controls and produces 3 PASS, 2 FAIL, 1 MISSING, and 1 WARNING. Its clearest example is a purchase order for 12,480 EUR and an invoice for 12,480 USD. The number matches, but the currency does not. PolicyProof shows the two exact source excerpts, reproduces the result, lets the reviewer change the policy threshold, and records the final decision with a required comment. PolicyProof is a review aid. It does not approve payments, certify compliance, authenticate source documents, or replace professional judgment.
How it works The system separates interpretation, verification, and decision-making. Importantly, the public demo runs deterministically, with no API key, so judges can reproduce the full workflow. Live GPT-5.6 was validated historically on the Northstar case and is documented in the repository, not executed live in the public demo. GPT-5.6 interprets and locates. In the optional server-side path, it proposes structured controls, extracts structured facts from fictional documents, and locates exact evidence excerpts. Zod validates model output. Unknown documents, malformed facts, and unsupported excerpts are rejected. TypeScript verifies objective rules. The deterministic engine evaluates amount, currency, date, threshold, evidence-presence, and role comparisons. A human decides. The reviewer confirms, rejects, edits, or accepts an exception with a required comment. The review is recorded. PolicyProof generates a Review Fingerprint and a locally verifiable receipt-integrity hash. These detect included-content changes, but they are not digital signatures, identity proofs, or trusted timestamps.
Who it is for PolicyProof is built for finance, procurement, internal-control, and audit teams that need faster reviews without losing traceability or human accountability.
How I used Codex and GPT-5.6 Codex accelerated architecture, implementation, debugging, test design, documentation, security review, and release preparation. I kept responsibility for the problem definition, domain boundaries, controlled scenarios, human-in-the-loop design, scope reductions, and final product decisions. GPT-5.6 is the interpretation and evidence-location layer. Codex was the development collaborator. Neither makes the final operational decision.
Reliability and testing The repository includes a provider-free evaluation harness and a release-verification workflow covering: • 3 controlled scenarios • 21 controlled conclusions • 34 controlled evidence references • exact-excerpt and evidence-to-control validation • threshold sensitivity and deterministic reproduction • Review Fingerprints and receipt-integrity modification detection • 7 isolated business-rule mutations • 10 named adversarial boundaries • 0 external network calls inside the scoped deterministic evaluation guard • 205 Vitest tests • 23 Playwright tests The harness distinguishes deterministic checks, mocked checks, and historical live evidence. It does not claim universal policy coverage or universal adversarial safety.
Main challenge The hardest part was preserving a trustworthy boundary between probabilistic interpretation and deterministic verification. A model can help interpret policy language and find candidate evidence, but unsupported excerpts, ambiguous references, and business-rule calculations must fail closed rather than being presented as facts. A second challenge was making traceability usable rather than merely available. PolicyProof offers both a focused demonstration and a full workspace while preserving the same engine and review state.
What I am proud of • Built an end-to-end evidence-led review workflow as a solo project. • Made every conclusion inspectable through exact source excerpts. • Kept GPT-5.6 outside the final deterministic control result. • Added reproducible fingerprints and locally verifiable receipt integrity. • Created a no-key deterministic path for judging and a separate server-side GPT-5.6 path. • Documented limitations explicitly instead of overstating compliance or cryptographic guarantees.
What I learned The strongest design decision was to assign one clear responsibility to each layer: GPT-5.6 interprets, TypeScript verifies, and the human reviewer decides.
What is next Future work includes broader policy domains, PDF parsing, collaborative review, and signed receipt formats. These are intentionally outside the current competition scope.
Built With
- codex
- gpt-5.6
- next.js
- openai-responses-api
- playwright
- react
- typescript
- vercel
- vitest
- zod