Inspiration

A rejection notice for a scholarship packet does not tell you what went wrong. It says the information was inconsistent, and that is the end of the conversation. There is no appeal, no clarifying email, no line number. Somewhere across three documents, two numbers disagreed, and a committee that read them side by side for the first time made a decision you cannot undo.

The maddening part is how small the error usually is. An applicant types 3.80 into a form from memory in March; the official transcript printed in January says 3.50. Household income goes on the aid application as $65,000; the tax statement says $48,000. Neither number is a lie. They were written months apart, by different people, on different paper, and nobody ever read both at once.

The information was already in the packet. Both numbers were right there. The only thing missing was someone checking them against each other while it could still be fixed.

So why not ask an AI? I tried, and the two failure modes showed up immediately. The first is hallucinated equivalence: ask a language model whether 3.80 and 3.50 match and it will happily explain that they are "approximately equivalent" or that the discrepancy is "likely a rounding difference." The second is missing evidence: you get a confident paragraph with no way to check where it read anything. For a document that decides whether someone can afford university, a tool that is usually right and never checkable is worse than no tool.

That led to the constraint the whole project is built around: the model is never allowed to decide whether two values match.

What it does

Parity cross-checks the three documents that carry the most clerical risk in an application packet — the application form, the official academic transcript, and the financial or tax statement — before you submit them.

One pass takes under two seconds on the sample packets, needs no account and no API key, and returns one of three verdicts per field:

  • Conflict — the values genuinely disagree
  • Needs Review — the engine refuses to guess
  • Consistent — the values match under deterministic normalization

Three documents cross-checked in under two seconds: 3 Conflicts, 2 Consistent, 0 Needs Review

Needs Review is the part people do not expect, and it is the most important state in the system. It fires when S. Sharma meets Suresh Sharma, because bureaucratic reviewers reject packets whose names do not match identity documents character for character, and guessing that an initial expands to a full name is exactly the risk an applicant is trying to avoid. It fires when two documents both read 12/08/2024, because when the day and month are both 12 or under with no written month name, Parity will not assume whether the issuing office writes day-first or month-first. Flip the date-format switch to your locale and the engine deterministically re-evaluates the same pair as Consistent.

Every verdict opens an Evidence Drawer rather than a summary. You get the verbatim quote the value came from, capped at 200 characters, the page it came from, and a coordinate overlay on the scan itself. Side by side, 3.80 GPA against 3.50 GPA, in the model's own returned text, so you are checking the document rather than trusting the tool.

Evidence Drawer showing 3.80 GPA quoted from the application form against 3.50 GPA from the official transcript, each with its verified region on the scan

Then you decide. You pick which document is authoritative — not the model — and a prioritized "Fix Before Submitting" checklist rewrites itself around that choice. Mark the transcript authoritative and Parity tells you to correct the form, never the other way round. It will not edit an official credential, and for conflicts that need the issuing office to act, it drafts a correction letter you can send to a registrar. You can export a PDF audit report or CSV for the administrative desk.

Prioritized fix list rewritten after marking the transcript authoritative: update the application form to 3.50 GPA

Two verification modes, with honest API costs: Fast mode is exactly 3 calls, one per document. Strict mode (the default) is 4 to 6 — it re-reads every field that would be labelled Consistent from a cropped band of the original scan, and only shows you a coordinate overlay if that second read agrees. If it does not, the overlay is withheld and the field carries a "Location not verified" badge, because a confidently wrong box drawn on a scan is worse than no box.

How we built it

The architecture is a hard split, enforced in the file layout rather than in a prompt instruction.

Perception. Google Gemini 3.1 Flash Lite reads the scans and does nothing else. A strict response schema (src/lib/gemini/schema.ts) constrains it to extracting a value, a verbatim quote from the page, a page number, and normalized bounding-box coordinates. It is never asked whether anything matches.

Decision. An isolated TypeScript engine (src/lib/compare/engine.ts) makes every judgment. Around it sits the normalization layer that turns messy international paperwork into comparable values: currencies across $ € £ ¥ ₹, income frequency annualized (weekly x52, biweekly x26, monthly x12) with the multiplier shown in the explanation, European dot-decimal grouping, ISO and DD/MM and MM/DD and textual dates, GPA on 4.0, 5.0 and percentage scales, and cultural name handling for accents, particles, compound Hispanic surnames, hyphenated British names and family-name-first ordering.

Containment. Before any comparison, src/lib/compare/containment.ts checks that the extracted value actually appears inside the quote the model returned. If the value is not literally in its own evidence, the field does not get to be Consistent.

The double read. src/lib/gemini/doubleRead.ts uses Sharp to crop the region the model pointed at and asks it to read that region again, independently. Only fields heading for a Consistent label are re-read, since those are the ones where a mistake is expensive.

Supporting it: Next.js 15 App Router, React 19, TypeScript, Tailwind CSS v3, jsPDF for reports, client-side downscaling to 1600px JPEG (1.5 MB per file, under 4 MB per request), in-memory IP rate limiting, and zero-PII logging. 109 Vitest unit tests, 2 Playwright end-to-end specs, and a benchmark harness (npm run eval:scans) over 14 raster scans generated by scripts/generate-scans.ts with synthetic camera blur, tilt and faded ink.

The planning came first, through the Devpost Learn Skill Pack: scope.md, prd.md and spec.md are in the repo, and the scope boundary they set — three document types, one packet, no fraud detection — is the reason this shipped as one coherent thing instead of six half-features.

Challenges we ran into

The bounding-box alignment trap. The first live double-read run was a disaster: 28.6% agreement, 4 of 14 checks, with 10 disagreements. I assumed extraction had failed. It had not. Gemini's vertical coordinates on dense tabular scans were drifting one or two rows, so the crop I sent back for verification showed Taxpayer: Alex Morgan instead of the Adjusted Gross Income row underneath it. I was re-reading the wrong part of the page and blaming the model for disagreeing with itself.

The fix was to stop trusting tight boxes: pad every crop to the full horizontal width of the document with one row of vertical padding, and batch all of a document's crops into a single multi-image call. Agreement went from 28.6% to 100.0% on the clean international scans, and secondary API calls dropped from 14 to 4 — a 71% reduction.

The part I keep coming back to is that the deterministic comparator caught all ten mismatches and flagged them Needs Review. The architecture did its job during its own worst failure. That is the strongest evidence I have that the split is real.

A bug that only the quote could reveal. Income period detection read the extracted value and missed the period. A payslip showing 15,000 with the quote ...15,000 per month was being annualized wrongly, because the word "month" lived in the evidence, not the value. The parser now inspects both.

Gemini being overloaded. Public endpoints return HTTP 503 and multimodal inference occasionally exceeded 25 seconds. Parity now does two exponential-backoff retries, falls back from gemini-3.1-flash-lite to gemini-2.5-flash, and when everything upstream is exhausted returns a clean 503 that points the user at the offline sample packets — which is why the demo works during an outage.

My own regional assumptions. The first build was quietly Indian: rupees, lakhs, "marksheet", local identity documents. For a tool aimed at people applying across borders that was a real defect, and fixing it meant a repository-wide sweep and a rule that ambiguity escalates to Needs Review rather than resolving to whatever my locale happens to be.

The build machine started corrupting files mid-project. Git objects failed to inflate and the test suite began segfaulting, while the same code passed elsewhere. The cause turned out to be NTFS metadata damage: a 148 MB native binary in node_modules had the exact same byte length as a known-good copy but a different checksum. Correct size, wrong contents — silent bit-rot. A filesystem repair and a clean clone fixed it, and it was an uncomfortably on-the-nose lesson for a project about documents that look fine and are not.

Accomplishments that we're proud of

Zero false-consistent errors. Across 9 distinct scan sets, of the 31 field pairs whose ground truth genuinely differs, 0 were ever labelled Consistent. Not once did differing documents get approved. That is the one metric where a failure actually hurts an applicant.

Reporting two metrics instead of one. A system that labels everything Needs Review would score a perfect 0% false-consistent rate and be useless, so true-consistent recall is published next to it: 11 of 14 agreeing pairs (78.6%) across all sets, 100% on the clean international scans. Both numbers, or neither means anything.

Publishing the 28.6%. The failed first double-read run is in the README with the fix beside it, because a benchmark that only shows the final number is marketing.

A demo that survives its own dependencies. Sample packets are precomputed, so the entire evaluation path runs offline in under two seconds with no key, no account, and no exposure to upstream quota.

Quote containment at 100% across all 47 extracted fields — every value provably present in its own evidence.

What we learned

Where you measure decides what you conclude. The bounding-box run told me extraction was broken. Extraction was fine; my verification crop was aimed at the wrong rows. One layer of measurement off, and the conclusion inverted.

Refusing to answer is a feature, not a gap. Every instinct says to resolve ambiguity and show a clean green check. The cases where Parity declines — initials, ambiguous dates, mismatched currencies, differing GPA scales — are the cases where being wrong costs an applicant a year.

Formatting differences are cultural, not erroneous. García Márquez, José versus Jose, 1.234,56 €, 12/08/2024. A tool that calls these mistakes is telling people their names are wrong.

Planning before prompting is what kept this coherent. The interview-first workflow produced a scope I could refuse features against. Most of what makes this feel like a product rather than a demo is the things scope.md said no to.

What's next for Parity

  • Real scans. Every benchmark document is synthetic. The honest next step is a ground-truth set of real, consented paperwork, and publishing whatever that does to the numbers.
  • More packet types. Visa applications and loan files have the same structure — employment payslips, tax assessments, sponsor identity — and the same silent failure mode.
  • Packets larger than three documents, including multi-page transcripts and appendices.
  • Locale auto-detection from document context, so the date-format switch becomes a confirmation rather than a question.
  • Accessibility work on the Evidence Drawer. The side-by-side comparison is the core of the product and it needs to work properly with a screen reader.
  • What it will never do: authenticate seals, detect forgery, or predict an admissions outcome. Parity checks whether your documents agree with each other. That boundary is deliberate.

Built With

Share this project:

Updates

Submission history