Inspiration

Last year my grandfather was taken to the emergency room five times because he had trouble breathing. Each time, a new doctor had to start from scratch.

The first time I was with him. I did not know his allergies or his blood pressure medicine. I did not know how long he had smoked or when he quit. I stood in the corridor phoning my mother and grandmother to piece it together for the doctor. By the fourth visit we had found a workaround. I called a doctor who had treated him twice before and asked him to brief the doctor on duty.

The last time I was out of the city. A relative went with him and could not explain his history well. The team did what emergency teams are trained to do and followed the standard protocol for breathing trouble. Protocols work. Still, a doctor who can see a patient's allergies and earlier ER reports can do better than a protocol. I believe this vital data gives the doctors a better chance.

It is not a rare story. In one Indian study only about 1 in 3 of the papers patients carried had the diagnosis, the medicines and the follow-up plan. In a German emergency department, 27% of patients arrived with critical history missing.

So I built Crata Health. It is a health record that belongs to the patient. A photo of a prescription or a short recording is enough to add a visit. Your confirmed allergies and medicines sit on an emergency card behind a QR code, and any doctor can open it in seconds.

How I built it

Crata is a web app. The backend is FastAPI with SQLite. The screens are React with TypeScript and Tailwind. Both ship in one Docker image on Render. Uploads are read in memory and never written to disk on the server.

Every visit goes through the same steps:

Read: Nemotron 3 Nano Omni on NVIDIA build reads document photos and scanned PDF pages. It also transcribes consult audio in pieces of about 30 seconds, cut at a quiet moment so no word is split. A PDF with a text layer needs no model at all.
Sort: Nemotron 3 Super sorts each page into a prescription, a bill, a lab report or a scan report. Bills get their own call so a pharmacy line never turns into a prescribed medicine.
Extract: Super on Nebius Token Factory runs at temperature 0. It returns each fact with the exact words it came from.
Check in code: Code looks for the quote in the source and drops the fact if it is not there. Code also matches each medicine against the patient's allergies using a table of about 30 drug families. It compares the bill with the prescription too. A brand the table does not know goes to Tavily, which finds the brand's own page on a trusted site.
Confirm: Allergies, medicines, diagnoses and events wait for the patient to confirm. Only confirmed entries reach the emergency card.

The rule behind step 4 is one line:

$$ \text{keep a suggestion} \iff \text{its quote} \subseteq \text{what the model read} $$

Every step has a backup on the other provider. When NVIDIA build is busy, Gemma 3 27B on Token Factory reads the photos. When one side is slow, Super answers from the other.

On top of the record sit two everyday screens. Before a visit turns "my head hurts since three days" into a short summary for the doctor. Ask about my health answers from your own record with charts, or from trusted health sites through Tavily. In both, code decides the policy. Code catches warning signs before any model call. It also picks the kind of doctor from a fixed list and filters any line that tells you to start or stop a medicine.

To know whether it works, I wrote 14 synthetic visits the app was never tuned on and ran each one 3 times:

What we measured	Result
Allergy clashes caught	15 of 15
False alarms	0
Billed but not prescribed, caught	12 of 12
Medicines read (none invented)	75 of 75
Same answer in all 3 runs	every case
Median time per visit	26.5 s
Token Factory cost for all 42 visits	16 US cents

The same cases as tilted, shadowed phone photos gave the same results. Per visit the cost works out to

$$ \frac{\$0.157}{42\ \text{visits}} \approx \$0.0037\ \text{per visit} $$

I built it over 20 days, one small piece at a time, each on its own branch with tests. The public code has 736 tests. I planned each piece, made the product calls and checked them by hand. Claude Code wrote most of the code.

Challenges I ran into

The free tier went quiet. On day one Nano Omni was listed on NVIDIA build, yet every call hung for 90 seconds. Later it answered with "503 Worker local total request limit reached (16/16)". I added a backup for every step. In the main eval the Gemma backup read

$$ \frac{16}{42} \approx 38\% $$

of the visits. Without it those visits would have failed.

One pipeline, two providers. Nano Omni is not served on Token Factory, and Token Factory has no audio model. So reading runs on NVIDIA build and reasoning runs on Token Factory. Recordings still have no backup.

Hindi numbers. On English speech Nano Omni caught 11 of 11 key facts, even with noise added. On a Hindi recording it caught 5 to 6 of 11. It wrote "chaiso pachas" for 650 and "Tindin" for teen din. In Hinglish it heard "sulfa drugs" as "salphad waks". That is an allergy. I chose to keep it as heard and let the patient correct it. A guess there could be worse than a typo.

Thinking ate the answer. With thinking on, Super used up its whole token budget and returned nothing. For reading photos, thinking off took 3 to 15 seconds against 36 to 128 with it on, and it read just as well.

Scans became diagnoses. I tested a CT and an MRI report and got 13 "diagnoses" to confirm. The prompt treated every Impression line as a conclusion. A new "scan report" kind fixed it. On synthetic copies the count went from 7 to 0.

Words that sound like advice. An early medicine summary said "you should stop the medicine". An early Ask summary called the printed range "normal". Code now drops any line that gives an order. Code also works out "higher", "lower" and "above the range" itself, and the model may only use those words.

Asking the wrong question. The first version of smart attach answered "none" on 3 of 3 tries. I had asked it to pick one test, and the lab report held three. The model was right and my design was wrong.

What I learned

Let the model propose and let code decide. The model reads well. It should not be the one deciding what is safe. Quotes, dates, who said what and which facts need the patient are all plain code. That made the app much easier to trust and to test.

Bigger is not always better. I ran all three Nemotron sizes on the same 42 visits:

	Nano	Super	Ultra
Clashes caught	14 of 15	15 of 15	15 of 15
Billed false alarms	5	0	9
Cost per visit (US cents)	0.18	0.37	1.54

Ultra read "Mox 500" on a bill as moxifloxacin when it was amoxicillin. That one slip caused all 9 false alarms, at about 4 times the cost of Super. Super was the only size with no misses and no false alarms.

Plan for the provider to fail. A model on a list is not a model that answers. Backups, short timeouts and a plain "Try again" for the patient mattered more than any prompt.

When the model says nothing, check your question. Smart attach taught me that one.

Fewer taps keep people careful. At first a visit needed 9 confirm taps. Now only the essentials wait. One tap takes the rest, and any row the code is unsure about still needs its own decision. If you ask a person to confirm everything, they stop reading.

Write numbers down. I kept a build log from day one with every model run, failure and timing in it. I never tuned a prompt on the eval cases. It also told me what I have not tested yet: handwritten prescriptions, which are the common case in India, and speech in Indian languages. That is where Crata goes next.