ProofTrail

An evidence-first requirements auditor that turns dense briefs into checklists and verifies whether each proposed supporting quote exists in the source. AI proposes. Evidence decides.

Comment

Inspiration

Students, hackathon teams, developers, freelancers, and proposal teams regularly work with dense project briefs, competition rules, and client specifications. Missing a single deliverable, eligibility rule, or deadline can disqualify a submission or derail a project.

While many people now use AI to summarize dense documents into checklists, a fluent AI summary can still include unsupported claims or repeat vague deadlines without surfacing uncertainty. I wanted to build a tool around a strict architectural boundary: the model is allowed to be creative, but it is never allowed to be the judge.

Or more simply: AI proposes. Evidence decides.

What it does

ProofTrail is an evidence-first requirements auditor that turns dense project briefs into an interactive, traceable requirements checklist.

It uses a split-screen workspace:

  • Left Panel (Source Document Viewer): Displays the original brief and highlights supporting quotations directly in context.
  • Right Panel (Requirements Audit Feed): Presents candidate requirements as interactive checklist cards with category and verification status filters, summary metrics, progress tracking, and localStorage session persistence.

Every candidate requirement is evaluated against the original source text and assigned one of three explicit verification outcomes:

  • Source Matched (Green): The proposed supporting quotation was verified in the original source text. Clicking the card scrolls to and highlights the exact passage in the source panel.
  • Needs Review (Amber): The supporting quotation exists in the source text, but a rule-based ambiguity check detected missing context that requires human judgment. For example, in the built-in sample, the candidate "Submit final deliverables by Friday before midnight" matches the source sentence "Final deliverables must be submitted by Friday before midnight through the Devpost portal." Because the calendar date and timezone are missing, ProofTrail flags it for human review rather than guessing.
  • No Source Match (Red): The candidate's proposed supporting quotation cannot be found in the source document. In the built-in sample, the candidate titled "Submit printed team affidavit by physical mail" proposes the supporting quote "All teams must mail a physical notarized affidavit to headquarters within 24 hours of registration." Because that sentence is absent from the source brief, ProofTrail rejects it as No Source Match and never fabricates a highlight.

Who it is for

  • Students and hackathon participants verifying competition rules, deliverables, and submission requirements.
  • Developers breaking down technical specifications into traceable tasks.
  • Freelancers, consultants, and proposal teams reviewing client briefs and requests for proposals (RFPs) where every requirement needs source traceability.

How I built it

I followed a plan-first workflow before writing application code, defining the problem boundaries and architecture in /devpost (scope.md, prd.md, spec.md, checklist.md, and an interactive architecture visual in app-map.html).

The application is built with:

  • Next.js 14 (App Router) and TypeScript (strict mode)
  • Tailwind CSS and Lucide React for a restrained, high-contrast developer-tool interface
  • Google Gemini SDK (@google/genai) using Gemini 2.5 Flash for optional live structured candidate extraction
  • Node.js native test runner for automated unit and route verification

Separating Generative Extraction from Deterministic Verification

The core engineering decision in ProofTrail is separating candidate extraction from verification:

  1. Candidate Proposal: In live mode, Gemini 2.5 Flash proposes structured candidate requirements along with their exact claimed supporting quotations. In the built-in offline demo, an 8-item illustrative sample fixture provides prepared candidates at runtime without requiring an API key (producing 5 Source Matched, 2 Needs Review, and 1 No Source Match).
  2. Deterministic Verification Kernel (lib/verifyQuote.ts): A separate TypeScript verification module checks whether each proposed quotation actually exists in the source text. Given identical inputs and rules, this kernel produces repeatable matching outcomes using literal matching, normalized matching for typographic quotes/dashes and multiline whitespace, original character-offset mapping, and rule-based ambiguity checks.
  3. Strict API Guardrails: Server-side API credentials are never exposed to the browser. If custom-document extraction is requested when no server-side Gemini API key is configured, the API returns a clear HTTP 503 response rather than silently falling back to unrelated sample data.

Challenges I ran into

  • Preserving Original Character Offsets During Normalization: Source documents often contain curly quotes, varied dash characters, or line breaks that cause naive substring checks to fail. However, if you strip or normalize whitespace to match a quote, the character indices shift—which breaks highlighting in the original document viewer. Solving this required mapping normalized comparison positions back to their exact character offsets in the raw source text inside lib/verifyQuote.ts.
  • Communicating Epistemic Boundaries Honestly: A matched quotation proves that the words exist in the source text, not that the candidate title's interpretation is 100% semantically complete. Similarly, "No Source Match" means the proposed quotation is absent from the source, not a formal proof that a rule could never be implied elsewhere. Designing the UI and status copy to reflect these boundaries without overwhelming the user took careful iteration.

Accomplishments that I'm proud of

  • A Focused, Test-Backed Verification Kernel: Building a dependency-light verification pipeline backed by 16 passing automated tests across 3 test suites (verifyQuote, analyzeRoute, and persistenceAndFiltering) with zero failures and a clean Next.js production build.
  • Zero-Key Offline Audit Mode: Designing the built-in sample audit so evaluators can immediately inspect how the runtime verification kernel handles green, amber, and red cases without needing to configure API credentials first.
  • Plan-First Execution: Translating the structured planning artifacts (scope.md, prd.md, spec.md, checklist.md, and app-map.html) directly into a cohesive, working proof of concept.

What I learned

I learned how valuable it is to separate AI-assisted extraction from deterministic software verification. Language models are effective at proposing structured candidates from unstructured prose, while deterministic TypeScript logic is much better suited for checking exact textual presence and surfacing rule-based ambiguities. Even when a quotation is present, human review remains essential for interpreting context and deadlines.

Known Limitations

As a focused proof of concept, ProofTrail has clear, intentional boundaries:

  1. It audits proposed candidates; it does not guarantee that 100% of requirements in a document were extracted.
  2. Quote verification confirms textual presence in the source, not perfect semantic interpretation.
  3. Ambiguity detection is rule-based and targets specific patterns (such as missing dates/timezones) rather than all possible semantic vagueness.
  4. The built-in demonstration audits prepared sample candidates at runtime rather than performing a live Gemini call.
  5. ProofTrail is an assistive inspection tool, not a guarantee of legal, regulatory, or contractual compliance.

Built With

Share this project:

Updates

Submission history