Inspiration

A listing promises one rent; an agreement states another. TwoTruths helps readers find passages to check when documents should agree.

What it does

TwoTruths puts original passages side by side for review. Each finding contains two verbatim passages checked against source text and offsets. This makes evidence inspectable, but does not make a finding correct. MiniLM retrieves related passages, DeBERTa checks possible contradictions, and rules compare values and conditions. Reviewers can confirm, dismiss, re-link or resolve findings, replay decisions and export literal evidence. It runs in the browser without an inference server, API key or document upload. First use downloads model assets. Findings are candidates; no findings does not establish agreement.

Evidence and limitations

On 12 hard held-out synthetic pairs, precision was 41.07%, recall 62.16% and F1 49.46%. All three fully consistent hard pairs produced false alarms. The hybrid did not show a statistically significant advantage over NLI-only. All evaluation documents and labels were AI-written, with same-model review and no independent human adjudication. The small synthetic corpus does not establish real-document or real-user reliability. Coverage budgets can omit comparisons; English only, no OCR. This is a reading aid, not legal advice. See the full held-out validation.

How we built it

Johan Vaz directed and reviewed AI coding agents OpenAI GPT-6.1 Sol and Anthropic Claude Opus 5.5, which wrote the application code, tests, synthetic evaluation data and documentation. The app uses React, TypeScript, Transformers.js, ONNX Runtime WASM and PDF.js. MiniLM and DeBERTa run in a browser worker; documents and corrections stay on the device. Fixed-template explanations and source-span checks prevent invented quotations, not incorrect findings.

Challenges and lessons

The initial hybrid underperformed a baseline. Development fixes addressed retrieval, evidence and quantitative scope; adversarial review found defects in matching, corrections, exports and progress reporting. The held-out results show that exact quotes do not guarantee correct conclusions. The NLI-evidence alternative scored higher on hard F1; selection was frozen before the held-out run.

Accomplishments and next steps

The working prototype combines real browser model inference with a persistent review loop and evidence exports. Next, gather independently adjudicated examples with permission, examine the simpler NLI-evidence approach on fresh development data, and improve coverage reporting.

Links

Live demo | Source | Captioned demo video and release | Direct video

Built With

Share this project:

Updates

Submission history