Eli

Inspiration

Every year, millions of people are discharged after surgery and sent home to recover and a lot of them, especially the elderly, go home alone. The most dangerous window in recovery isn't the operating room; it's the first week at home, where a missed antibiotic dose, a small fever, or a fall can quietly turn into a readmission. Nurses can't call every patient every day, and family members are often a plane ride away and left in the dark. We wanted to give every discharged patient a warm voice that checks in on them — and give their loved ones peace of mind without either of them lifting a finger.

What it does

Eli is a post-op voice agent. It phones patients on their critical recovery days (day 1, 3, 7…), asks in a calm, unhurried voice how they're feeling — pain, the incision, medications, dizziness, mood — and listens. After the call, it turns the conversation into a clinical summary with triaged flags (e.g. "missed evening antibiotic," "pain still 6/10," "feeling lonely") and texts a plain-language recap to the patient's loved ones, escalating when something sounds urgent.

Patients can also call Eli anytime with questions. Because it knows their chart, it can answer the things elderly patients actually ask — "Which one is my red pill, and when do I take it?" — by matching the description to their medication list and using the current time to say whether a dose is due now or later.

Behind it is a clinician dashboard: patient roster with live triage badges, an editable EHR (notes, medications with "looks like" identifiers, loved-one contacts), and every call's transcript + key findings.

How we built it

• Telephony + voice: Twilio Programmable Voice bridged to ElevenLabs Conversational AI over WebSockets.
• Agent as a workflow: we configured ElevenLabs' subagent workflow nodes entirely through their management API — identify → recovery check → (urgent → escalate) → wrap up — and made it direction-aware so an inbound caller gets "how can I help?" instead of a check-in.
• Post-call intelligence: on hangup we pull the authoritative transcript from ElevenLabs, run a Claude-powered analysis (with a keyword-heuristic fallback) to produce a summary + triage level + flags.
• Stack: FastAPI + async SQLAlchemy + Neon Postgres on Render; Next.js dashboard on Vercel.

Accomplishments that we're proud of

• A real phone call to a real number where Eli greets the patient by name, runs a genuine recovery check, and — from a live transcript — surfaces that they missed their evening Amoxicillin and are feeling lonely.
• The "which is my red pill?" feature actually works, because we modeled medications the way patients describe them, not the way pharmacists do.
• Reshaping a production voice agent's workflow via API instead of clicking through a dashboard.
• A clinician dashboard that turns a fuzzy phone call into structured, triaged, shareable findings.

What we learned

• With voice agents, latency and reliability live in the details — one disallowed config field or one missing variable is the difference between a conversation and silence. Deep logging beats guessing.
• Design for how people actually talk. Elderly patients don't say "Amoxicillin 500mg"; they say "the small red one." Meeting them there was the whole product.
• Keeping the EHR as a single source of truth that feeds both the outbound check-in and inbound Q&A kept the system coherent as it grew.

What's next for Eli

• Automated scheduling of check-ins on each patient's critical days (today it's demo-triggered).
• Trend tracking across calls — watch pain go 9 → 6 → 3, or a flag clear, right on the patient timeline.
• EHR integration (FHIR) so Eli reads real discharge instructions instead of a hand-entered chart.
• Clinician-tunable escalation and on-call handoff for urgent findings, plus multilingual support for the patients who need it most.

Built With

• fastapi
• nextjs