Inspiration
In Bangladesh most medicine is paid for out of pocket, and a prescription is a handwritten slip of brand names with a 1+0+1 dose code. Families often can't read it. Nobody tells them that the same medicine is sold by dozens of companies at very different prices, or that two slips on one table can hold the same ingredient twice. We asked the question every family asks at the pharmacy counter: ডাক্তার কী লিখলো? What did the doctor write?
What it does
Take a photo of a prescription. Likhlo's agent reads every handwritten line, then for each medicine tells you:
- what it is (generic, strength, maker) and what it is for, in English and Bangla
- when to take it: morning, noon or night, before or after food, for how many days
- every brand of the same generic, strength and form, with its price, and the total at the lowest price
- when two medicines clash: a shared ingredient (a double dose), or a warning in one medicine's leaflet about another, quoted word for word
On Rahima Khatun's sample prescription it catches that Napa Extend and Ace Plus are both paracetamol, quotes the clopidogrel leaflet ("Avoid concomitant use of omeprazole or esomeprazole") against her Seclo, and prices the same medicines at ৳409 instead of ৳612. On Abdul Karim's monthly diabetes and blood-pressure list it is ৳798 instead of ৳1,675: about ৳10,500 a year.
You can also: listen to the summary in Bangla, save a one-page slip for the pharmacy (or send it on WhatsApp), add daily reminders to your phone, and search any medicine by name with no photo at all.
How we built it
One Gemini agent with tools, built on the Vercel AI SDK. Gemini reads the photo (vision) and decides what to look up; every fact it shows comes back from a tool over 21,714 Bangladeshi medicine brands and 1,570 generic leaflets:
- record_lines: reads each line, its dose code, food timing and days, with its position on the photo
- search_medicine: fuzzy search, so a slip of the pen ("Seclo 2O") still matches
- choose_medicine: picks the brand whose name, strength and form fit; prices and the clash check come back
- check_clashes: shared ingredients and leaflet warnings, quoted verbatim; a sentence that says "no interaction" is never shown as a warning
- read_leaflet: for a child, pregnancy or kidney disease
- finish: plain explanations in English and Bangla
Doses, prices and totals are computed in code from the data, never by the model, and the model's text is filtered so it can never tell anyone to stop or change a medicine. The agent's steps stream to the page and are drawn on the photo as it works. Next.js on Vercel; Gemini TTS for the Bangla audio; a model fallback chain keeps live readings working when one model is out of quota.
Devin built four pull requests: the pharmacy slip and animated savings number, the medicine search, phone reminders, and the browser tests with CI on GitHub.
Challenges we ran into
- A leaflet sentence like "No clinically significant drug interactions have been found with azithromycin" first showed up as a warning. We now skip negated sentences and test it.
- A fallback model read "1+0+1 (খাবার পর) - ৫ দিন" as five doses a day. The dose is now taken only from the 1+0+1 pattern, with bounds, and golden tests check every recorded reading against the data.
- Free model quotas are small, so the sample buttons replay recorded real runs, and the search works with no AI call.
Accomplishments that we're proud of
- Every price, dose and warning on screen is checked against the data by 73 automated tests, plus browser tests in CI.
- It works in Bangla, the way families actually talk about medicine.
What we learned
The model is best at reading messy handwriting and deciding what to look up; the facts a family acts on should come from data, quoted and checkable.
What's next
Read the medicine strip the pharmacy hands over and check it matches the prescription; add official price updates; work with pharmacies and community health workers.
Built With
- claude-code
- devin
- gemini-tts
- github-actions
- google-gemini
- next.js
- playwright
- react
- typescript
- vercel
- vercel-ai-sdk
- vitest
