Inspiration
Medication errors from bad handwriting are a well-known, preventable harm. Mix-ups like Celebrex vs. Celexa, QD read as QID, or .5 mg read as 5 mg hurt patients every year. We wanted something a patient or pharmacist could point a phone at to get a second opinion before a misreading causes harm.
What it does
Script Doctor is an Android app. You photograph a handwritten prescription and it:
- Transcribes each medication: drug, strength, form, route, frequency, duration and quantity.
- Explains the directions in plain language.
- Rates its confidence for each medication and lists other plausible readings.
- Runs safety checks on the device: ISMP error-prone abbreviations, trailing or missing leading zeros, look-alike/sound-alike drugs, high-alert drugs read with low confidence, and missing strength or frequency.
- Sums it all up as Low risk, Verify before use, or High risk: confirm with prescriber.
How we built it
- Kotlin Multiplatform and Compose Multiplatform. Nearly all the code is shared in
commonMain. - Images are downscaled to 1568 px or less with EXIF-aware rotation, then sent to Gemini's vision model using a strict JSON schema for the output and high thinking.
- The safety rules are deterministic code, not AI, so the main warnings don't rely on the model.
- History is stored only on the device. Requests are sent with
store: false.
Challenges we ran into
- Getting the model to admit uncertainty instead of guessing with confidence. We fixed it with per-medication confidence scores and a list of alternative readings.
- Calling the API over plain HTTP with Ktor, because no platform SDK works in shared code.
- Cutting scope mid-hackathon. We dropped iOS to focus on Android and switched model providers part way through.
Accomplishments that we're proud of
- Safety checks that still catch dangerous patterns when the AI reads the script correctly.
- An honest UI that shows uncertainty instead of hiding it.
- A working, installable APK, with tests for the safety rules and the API client.
What we learned
- In healthcare AI, showing how confident the model is matters as much as being accurate.
- Simple fixed rules and an LLM work well together: the model reads the handwriting, the rules catch the known dangers.
- Structured outputs make an LLM's answer reliable enough for an app to depend on.
What's next for Script Doctor
- Route requests through a backend so no API key is stored on the phone.
- Check against a real drug database such as RxNorm for interactions and dose ranges.
- Bring back iOS.
- Test with pharmacists on real, consented prescriptions.
- Support more languages for the plain-language directions.