NaniAI

Healthcare goes quiet after the prescription. NaniAI carries the whole episode labs, waiting, results and only nudges you when something changes.

Comment 1

Inspiration

Healthcare doesn't end when you walk out of the doctor's office it just goes quiet. https://www.youtube.com/watch?v=7yFRswYJv3M

You leave with a prescription and a list of tests, and suddenly everything is on you: finding a lab, arranging the tests, waiting for reports, remembering what your previous results looked like, and figuring out whether anything has actually changed.

The problem isn't that patients don't care. It's that nobody is continuously holding the thread of a single care episode.

We kept coming back to one word: Nani — grandmother.

The person in a family who remembers your history without being asked, checks in without making it a big deal, and notices when something is different from last time.

We wanted to build an agent that behaved like that: not a chatbot you have to keep prompting, but a system that can carry a care episode forward across days and only speak up when something actually deserves attention.

What it does

A patient starts by uploading a doctor's prescription — even a handwritten one.

From there, Nani AI carries the episode forward:

  • Reads the prescription and identifies the investigations that were ordered.
  • Classifies tests by urgency and estimates how confident it is in the extraction.
  • Continues automatically when confidence is high, asks the patient to verify the extracted tests when confidence is uncertain, and stops for human review when the input is too unreliable.
  • Finds nearby diagnostic labs and selects a practical option.
  • Sends real booking-request emails and creates a tentative calendar hold.
  • Stores the episode and waits — potentially for hours or days — without requiring the patient to keep the application open.
  • Detects when a new lab report arrives.
  • Extracts the reported values together with the reference ranges printed on that specific report.
  • Compares those results with the patient's previous history.
  • Flags a meaningful change and requests a follow-up consultation only when necessary.
  • Closes the episode quietly when the results are stable.

The goal isn't to return another document summary.

It is to take a multi-step process that normally depends on the patient remembering every next step and carry that process forward safely on their behalf.

How we built it

Nani AI is built around three specialist agents using the Strands Agents SDK with foundation models through Amazon Bedrock.

Intake agent

The intake specialist reads prescription images and PDFs, including handwriting, and extracts the tests, medicines and relevant context.

But extraction does not automatically mean action.

Nani AI uses confidence-gated autonomy:

  • High confidence: continue automatically.
  • Medium confidence: enter AWAITING_CONFIRMATION and show the extracted tests to the patient for verification.
  • Low confidence or no reliable tests: stop and move to NEEDS_HUMAN.

The confirmation screen lets the patient edit, remove or change the urgency of an extracted test while comparing it directly against the original prescription.

Logistics agent

Once the tests are confirmed, the logistics specialist uses the Google Places API to find diagnostic labs near the patient and reasons about which option is practical.

It then performs real external actions:

  • sends a booking request through Amazon SES
  • creates a tentative hold using Google Calendar

We deliberately call it a booking request, not a confirmed appointment. Most diagnostic labs do not expose public booking APIs, so pretending otherwise would make the demo look better while making the product less honest.

Diagnostics agent

When a report arrives, the diagnostics specialist reads it and extracts the measured values together with the laboratory's own printed reference ranges.

Those results are compared against the patient's previous results stored in Amazon DynamoDB.

The agent then evaluates whether the change is meaningful enough to flag.

If it is, Nani AI can request a follow-up consultation and generate a plain-language summary.

If nothing meaningful has changed, the episode closes without creating unnecessary work.

The architecture

The agents provide judgement, but they do not control the workflow.

A deterministic Python coordinator owns the care-episode state machine and validates every legal transition before moving forward.

That distinction is important.

An LLM can answer:

  • What tests appear on this prescription?
  • Which nearby lab is practical?
  • Has this result meaningfully changed?

But ordinary application code decides:

  • Is this transition allowed?
  • Has the patient confirmed the tests?
  • Has this action already been performed?
  • Should the episode continue, wait, retry, escalate or close?

In short:

Agents provide judgement. The application owns control flow.

The backend runs on AWS Lambda behind Amazon API Gateway.

Amazon DynamoDB stores episode state, patient history, profiles, timelines and idempotency claims.

Amazon S3 acts as the report inbox.

When the system is waiting for a report, nothing needs to stay running.

Amazon EventBridge Scheduler periodically wakes the backend, which finds episodes ready to progress and resumes them from their stored state.

AWS Secrets Manager protects integration credentials, while Amazon SES handles booking and consultation emails.

This gives Nani AI two fundamentally different triggers:

The patient starts the episode. The scheduler carries it forward.

That is what makes it more than a document-processing application.

Challenges we ran into

The hardest part was waiting

Most agent demos begin with a prompt and finish a few seconds later.

A real care episode can take days.

We had to design the entire workflow so that no process needs to stay alive while the patient takes a test or waits for a report.

Every important step is persisted before the next one begins. If the runtime disappears after any action, the next invocation can reconstruct the episode from DynamoDB and continue from the correct state.

Preventing duplicate real-world actions

Schedulers retry. Clients retry. Networks fail.

An autonomous system cannot respond to that by sending two lab-booking emails.

Before any external side effect, Nani AI creates an idempotency claim using an atomic DynamoDB conditional write.

Only the first invocation succeeds.

If EventBridge fires the same workflow twice, the second invocation sees that the action has already been claimed and skips it.

This turns retries from a dangerous edge case into a normal part of the system.

Reading handwritten prescriptions safely

Handwritten prescriptions were one of the most difficult inputs.

The model could often identify the test list reliably while still being overconfident about secondary information such as dates or examination findings.

That led us to introduce the confirmation gate instead of choosing between two bad extremes: asking the patient to confirm everything or blindly trusting every extraction.

The system can now act when confidence is strong, ask when uncertainty is manageable, and stop when it cannot safely proceed.

Making the agent do nothing

Another surprisingly difficult problem was teaching the system not to overreact.

An autonomous healthcare assistant that recommends a consultation after every report is not useful.

Stable results should end with silence.

Getting that behaviour right — distinguishing a meaningful change from something that does not require action — became one of the most important parts of the product.

Real healthcare integrations are messy

There is no universal Indian diagnostic-lab booking API.

Instead of simulating a fake successful booking, Nani AI sends a genuine booking-request email and creates a tentative calendar hold.

Building around the limitations of the real world rather than pretending they do not exist made the workflow more credible.

Accomplishments that we're proud of

We built an agent that survives beyond the original request

Nani AI doesn't end when prescription extraction finishes.

An episode can stop at AWAITING_REPORT, remain dormant, and later resume when a new report arrives — without another prompt from the patient.

Three specialists with deterministic orchestration

Intake, logistics and diagnostics each perform a specialised reasoning task through Strands and Amazon Bedrock.

But no model is allowed to arbitrarily move the medical episode through its state machine.

That separation gives us the flexibility of agents without giving an LLM unrestricted control over the workflow.

Confidence-gated autonomy

Nani AI doesn't treat every model response as equally reliable.

Strong input can proceed autonomously.

Uncertain input gets patient confirmation.

Unsafe input stops.

For us, this is a more meaningful definition of autonomy than simply allowing the model to do more things.

Atomic protection against duplicate actions

Our DynamoDB idempotency design means multiple scheduler invocations can safely process the same episode without sending duplicate booking or consultation requests.

For an autonomous system performing real external actions, that reliability matters as much as the model itself.

Real external actions

The application does more than generate recommendations.

It searches real locations, sends real emails, creates calendar holds, receives documents through S3 and resumes episodes automatically through EventBridge.

We designed the normal outcome to be quiet

One of our demo patients has a meaningful worsening trend and gets a follow-up request.

Another has stable results and simply reaches CLOSED.

Both are successful outcomes.

Nani AI is not rewarded for taking more actions. It is designed to take the right action.

What we learned

Multi-day agents are primarily state-management systems

The model is only one part of the problem.

Once an agent has to survive for days, reliability depends on durable state, retries, event triggers, idempotency and resumability.

The difficult question stops being:

"Can the model understand this document?"

and becomes:

"Can the whole system safely remember what has already happened and know what should happen next?"

Deterministic orchestration works well with LLM specialists

We found a useful architectural boundary:

Use models for ambiguity. Use code for guarantees.

Models are excellent at reading handwriting, interpreting reports, choosing between practical options and explaining results.

Legal state transitions, retries and side-effect protection are better handled by deterministic software.

Structured output matters more than clever prompts

Explicit schemas made prescription and report extraction dramatically easier to validate and consume.

Instead of trying to make prompts increasingly elaborate, constraining what the model returned made the rest of the system simpler and safer.

Doing nothing is a real agent decision

Autonomy is often demonstrated by how many tools an agent can call.

Healthcare taught us the opposite lesson.

Sometimes the correct autonomous decision is:

Nothing has meaningfully changed. Close the episode.

Idempotency must come before side effects

It is not enough to send an email and then record that it happened.

The system has to claim the action first and only then perform it.

That ordering becomes critical as soon as autonomous systems interact with the real world.

Good autonomy has boundaries

We started thinking about autonomy as the ability to continue without a human.

We ended thinking about it as the ability to know when a human is necessary.

AWAITING_CONFIRMATION and NEEDS_HUMAN are therefore not failures of autonomy.

They are part of making autonomy trustworthy.

What's next for Nani AI

Better cross-report normalization

Different laboratories may label the same test differently.

Normalising codes such as PLT, Platelet Count and PLATELETS into a canonical representation would make long-term trend matching substantially more reliable.

A patient-facing episode brief

We want to combine the prescription context, tests performed, results and eventual outcome into a single understandable episode summary.

Not a diagnosis — simply enough context for the patient to understand what happened before their next consultation.

Deeper lab integrations

Where diagnostic providers expose APIs or partnerships are available, Nani AI could move from sending booking requests to checking real availability, receiving confirmations and automatically connecting reports to the episode.

Notifications and caregiver access

Future versions could notify patients when reports arrive or when follow-up is recommended and allow trusted family members or caregivers to follow the same episode.

That fits the original idea behind Nani AI: someone quietly remembering the thread when life gets busy.

India-specific healthcare integrations

Longer term, integrations with ABDM/FHIR-compatible patient records and major diagnostic providers could allow the system to build a more complete longitudinal view without requiring repeated manual uploads.

Production-grade clinical safety

Before use beyond a hackathon environment, the system would need stronger identity controls, clinical-review workflows, auditability, privacy controls and clearly defined escalation procedures.

Our goal remains the same:

Nani AI should summarise, compare and flag — never pretend to replace the clinician.

Built With

Share this project:

Updates

Submission history