-
-
A review decision is saved alongside source selection and an explanatory note.
-
Highlighted passages in the synthetic rental listing and tenancy agreement.
-
Hard held-out synthetic evaluation compares the hybrid with baseline methods.
-
Browser model progress while comparing the synthetic rental example.
-
TwoTruths landing page with two document inputs and the rental example.
Inspiration
A listing promises one rent; an agreement states another. TwoTruths helps readers find passages to check when documents should agree.
What it does
TwoTruths puts original passages side by side for review. Each finding contains two verbatim passages checked against source text and offsets. This makes evidence inspectable, but does not make a finding correct. MiniLM retrieves related passages, DeBERTa checks possible contradictions, and rules compare values and conditions. Reviewers can confirm, dismiss, re-link or resolve findings, replay decisions and export literal evidence. It runs in the browser without an inference server, API key or document upload. First use downloads model assets. Findings are candidates; no findings does not establish agreement.
Evidence and limitations
On 12 hard held-out synthetic pairs, precision was 41.07%, recall 62.16% and F1 49.46%. All three fully consistent hard pairs produced false alarms. The hybrid did not show a statistically significant advantage over NLI-only. All evaluation documents and labels were AI-written, with same-model review and no independent human adjudication. The small synthetic corpus does not establish real-document or real-user reliability. Coverage budgets can omit comparisons; English only, no OCR. This is a reading aid, not legal advice. See the full held-out validation.
How we built it
Johan Vaz directed and reviewed AI coding agents OpenAI GPT-6.1 Sol and Anthropic Claude Opus 5.5, which wrote the application code, tests, synthetic evaluation data and documentation. The app uses React, TypeScript, Transformers.js, ONNX Runtime WASM and PDF.js. MiniLM and DeBERTa run in a browser worker; documents and corrections stay on the device. Fixed-template explanations and source-span checks prevent invented quotations, not incorrect findings.
Challenges and lessons
The initial hybrid underperformed a baseline. Development fixes addressed retrieval, evidence and quantitative scope; adversarial review found defects in matching, corrections, exports and progress reporting. The held-out results show that exact quotes do not guarantee correct conclusions. The NLI-evidence alternative scored higher on hard F1; selection was frozen before the held-out run.
Accomplishments and next steps
The working prototype combines real browser model inference with a persistent review loop and evidence exports. Next, gather independently adjudicated examples with permission, examine the simpler NLI-evidence approach on fresh development data, and improve coverage reporting.
Links
Live demo | Source | Captioned demo video and release | Direct video