DayOne-Équipe DDRA

What it does

DayOne is a WhatsApp-style assistant for midwives. She photographs a page of the paper maternal register, and DayOne turns it into a structured, encrypted record that she reviews field by field and links to the right patient across visits.

Offline-first. Photos are encrypted on the phone (AES-GCM, key derived from her PIN) and saved before any network call. When the network returns, the queue replays in capture order.
Never hides doubt. Each field is known, unknown, empty, illegible or not applicable, never a vague "N/A". Doubts are listed first.
Numbered corrections. "Corriger" lists every field: one number, one new value.
No silent overwrites. Re-sending a photo shows the page already saved and asks: keep it or reprocess.
She decides the patient match. Nothing is created automatically when a plausible match exists.
If the AI is down, she can retry or fill the page by hand.

How we built it

A TypeScript monorepo: shared pure logic (schemas, validators, state machine, patient matching), a Hono + SQLite server using Gemini vision, a React PWA with an IndexedDB queue and local encryption, and an evaluation harness.

Schema-guided extraction: a vision model reads each photo into eight typed page schemas (dates, bounded integers, checkboxes, visit tables) instead of returning free text.
Confidence is recomputed: deterministic validators (due date, parity, blood pressure) can downgrade the model's own confidence.
Privacy by construction: no schema has a field for name, ID, phone or address.
Strict 12-state lifecycle: skipping patient matching is impossible by construction.
Blurry photos are refused on the device, before anything is sent.

Challenges we ran into

No ground truth existed, so we corrected 24 pages by hand, checkboxes included.
Our first evaluator was wrong: a perfect extraction scored only 82.8 %. We fixed it, added a test, and dropped the old figure.
Silent checkbox errors: the model read marked boxes as empty with 0.9 confidence. We added a second targeted reading that flags disagreements, validated on only 3 pages.
Reconnection race: several triggers sent every page twice. We serialised the queue.
The API was our limiting factor: a free-tier key, a 500-request daily quota we exhausted, and 503 overloads. We built the evaluation around a cache so we could re-score without spending calls.

Accomplishments that we're proud of

On 24 synthetic pages (759 fields), clean pages reach 97.8 % accuracy with 1.3 % silent errors (wrong and marked sure), up from 95.3 % and 3.7 %. Among fields returned with confidence ≥ 0.9, 97 % are correct.
We measure silent errors, the failures that matter for safety, not just accuracy.
299 automated tests, all green.
A real offline flow, not a demo trick: nothing is lost if the connection drops.

What we learned

Measure the failures that matter, not just the headline accuracy.
An honest "I'm not sure" is a feature, and the review flow matters as much as the model.
Deterministic checks around a probabilistic model are cheap and effective.
Be careful with your own numbers: our evaluator was wrong before our model was.

What's next for DayOne

Sync the verified record. Today, "Synced" does not carry the midwife's corrections; they stay on the device. This is the main remaining work.
Ground truth for real photos, and a measured accuracy on handwriting.
Authentication, encryption at rest, and automatic blurring of the name zone on stored images.
Test stronger models with a paid quota. We expect better results, but we have not measured it, so we claim no number.
A real WhatsApp channel (only api-client.ts needs replacing).