NaniAI

Landing
Dashboard
Particular Episode
Episodes Insights
patient profile
Episode Result
Architecture

Inspiration

Healthcare doesn't end when you walk out of the doctor's office it just goes quiet. You're handed a prescription with a list of tests, and from that moment on, everything is on you: finding a lab, booking it, waiting days for results, remembering what your last report said, and noticing if something's actually changed. Most people don't do any of that well, not because they don't care, but because nobody is actually watching the whole arc of a single care episode.

We kept coming back to one word: Nani. Grandmother. The person in a family who remembers your history without being asked, checks in without making it a big deal, and knows the difference between "you're fine" and "this is different from last time." We wanted to build an agent that behaved like that not a chatbot you have to keep prompting, but something that stays with you across days and only speaks up when it actually matters.

What it does

You upload a doctor's prescription even a handwritten one. From there, NaniAI carries the entire care episode forward on its own:

Reads the prescription and identifies which tests were ordered, classifying each as urgent or routine Finds nearby diagnostic labs and sends real booking request emails plus tentative calendar holds Waits genuinely waits, across days, without needing to be reopened or re-prompted Nudges labs that haven't responded, on a schedule, entirely on its own Reads the lab report the moment it arrives, extracting every value along with the reference ranges printed on the report itself Compares the new results against the patient's full history to see what's actually changed Books a follow-up consultation only if something meaningfully changed and sends a plain-language summary either way Stays quiet when nothing has changed, because most of the time, that's the right response

The output isn't a summary of a document. It's a decision, carried out, at the end of a multi-day process nobody had to babysit.

How we built it

NaniAI is a coordinated set of agents built on Google's Agent Development Kit (ADK), sitting on top of Gemini via Vertex AI:

A root coordinator doesn't do any work itself it reads the current state of a care episode from Firestore and decides which specialist needs to run next. An intake agent reads the prescription (Gemini 3.5 Flash, natively multimodal no separate vision model needed) and classifies each ordered test as urgent or routine. A logistics agent finds nearby labs through the Places API, ranks them on distance, rating, and open hours, and sends real booking requests through Gmail and Calendar. A diagnostics agent reads the incoming lab report, extracts values against the reference ranges printed on the report itself, pulls the patient's prior history from Firestore, and hands the one genuinely hard decision is this change clinically meaningful to Gemini Pro.

The entire thing runs on Cloud Run, scaling to zero between actions, woken by Cloud Scheduler on a tick that checks Firestore for episodes waiting on something. Every state transition is written before the next step begins, so if the container dies mid-episode, the next wake-up picks up exactly where it left off. Every booking request carries an idempotency key, written before the send happens, so a scheduler firing twice never becomes a duplicate email.

Challenges we ran into

There's no way to actually "book" a slot at an Indian diagnostic lab. No public API exists. We had to be honest about this rather than fake it NaniAI sends a real request email and holds a real calendar slot, and we frame it exactly that way instead of pretending it's a confirmed booking.

Making the agent do nothing was harder than making it act. The easy failure mode was an agent that flags everything. Getting the diagnostics agent to correctly stay quiet on a normal, stable result and only escalate on a genuine trend took more prompt and decision-boundary work than the extraction itself.

Surviving the wait. Cloud Run holds no memory between invocations, so the entire multi-day episode had to be representable as pure state in Firestore, resumable from any point, with nothing assumed to still be "in progress" in memory.

Reading handwritten prescriptions reliably. This was the case we cared about most and the hardest to get consistent tightening the extraction prompt with an explicit output schema mattered more than anything else we tried.

Accomplishments that we're proud of

We shipped a real multi-day agent, not a demo script. A prescription upload on live Cloud Run runs through intake → lab search → real Gmail booking requests and Calendar holds → AWAITING_REPORT, with every step written to Firestore before the next one starts. Cloud Scheduler wakes waiting episodes, nudges labs, and can pick up delivered reports from a Cloud Storage inbox — no re-prompting, no assumed in-memory state.

We proved the riskiest assumption first: Gemini 3.5 Flash reads real Indian prescriptions, including handwriting, with structured output doctor, medicines with dosing shorthand (1-0-1, H/S), tests, and urgency without inventing a diagnosis when none is written. We verified it was visual handwriting recognition, not OCR passthrough.

Three genuine ADK specialists, one deterministic coordinator. Intake and diagnostics run as multimodal ADK agents with fixed schemas; logistics is tool-using (Places API) and reasons about distance, hours, and ratings. The LLM never chooses state transitions the coordinator does so episodes are safe, idempotent, and resumable after a cold start or crash.

Real external actions, honestly framed. Because no Indian diagnostic lab exposes a booking API, NaniAI sends real request emails and tentative calendar holds instead of faking confirmations. Idempotency keys are claimed in Firestore before each send, so a scheduler firing twice cannot double-book.

The hard part was staying quiet. On a real multi-page lab report, extraction pulled values against printed reference ranges, compared against history, and Gemini Pro correctly returned “all clear, no consult” no over-flagging. Escalation is reserved for genuinely meaningful change.

Full stack live on GCP: Firebase-hosted frontend wired to the Cloud Run backend, three demo patient profiles with different histories, an episode timeline the UI renders step-by-step, and device geolocation so lab search uses the patient’s actual location when available.

What we learned

Structured output beats clever prompting. Tightening extraction with explicit JSON schemas (response_schema / Pydantic) did more for consistency on handwritten prescriptions and multi-page lab reports than chasing a bigger model.

The best architecture split is: deterministic orchestration + LLM specialists. Medical episodes need legal transitions, retries, and idempotency. Letting the model “decide what happens next” would be unsafe; letting it read documents, pick a lab, and judge significance is exactly what it’s good at.

“Do nothing” is a feature, not a gap. The failure mode we feared most was an agent that flags everything. Calibrating the significance step routine vs. urgent, consult vs. close took more iteration than extraction itself.

Multi-day agents are state problems, not model problems. Cloud Run forgets everything between invocations. The entire episode had to live as pure JSON in Firestore state, timeline, patient history, idempotency keys so any tick can resume from exactly the right point.

Indian healthcare has real integration limits. No public lab booking API, reference ranges printed on reports (not a separate knowledge base), and location that often means “nearest lab to where you actually are,” not a profile city. Building honestly around those constraints made the product stronger than pretending APIs exist.

Model routing is part of the engineering story. Bulk extraction runs on Gemini 3.5 Flash; the single hardest decision (is this change clinically meaningful?) goes to Gemini 3.1 Pro Preview. We learned to route deliberately rather than defaulting everything to one model.

What's next for NaniAI

Smarter history matching. Lab reports use inconsistent test codes run-to-run (PLT vs PLATELETS). Canonical normalization would make cross-report trends reliable before we scale beyond demo patients.

A patient-facing episode brief. Synthesise prescription context + lab findings into one plain-language “here’s what this episode was about” summary not diagnosis, but enough for a patient to walk into a follow-up informed.

Deeper lab follow-through. Parse lab reply emails, WhatsApp nudges for non-responsive labs, and tighter inbox automation so the wait phase feels as agentic as intake still honest about “request sent” vs. “slot confirmed.”

Real patient accounts and notifications. Auth, push/email when a report lands or a consult is recommended, and family/caregiver views the “Nani remembers your history” story works better when it follows you across devices.

India-specific rails. ABDM/FHIR-aware history import, integration with major lab chains where partnerships exist, and optional device vitals from the patient profile feeding trend context over time.

Production-grade safety. Clinical review workflows for NEEDS_HUMAN, audit logs, and guardrails before anything touches real patients outside a hackathon demo.