Phi Medical

An AI voice and text assistant for post-op and clinic follow-up. It understands tone, answers patient calls/messages, keeps summaries for clinicians, and routes urgent cases to the right specialist.

Comment

Inspiration

Post-op and routine follow-up calls eat up clinic time, yet patients still feel anxious and unheard between visits. We wanted something that sounds human, listens like a nurse, and never forgets a detail—all without another app to download. The idea: a voice-and-text companion that checks in, captures concerns, and keeps staff instantly in the loop.

What it does

Phi Medical is a phone- and SMS-based follow-up assistant:

  • Speaks and texts naturally — patients can call or reply to a message, whichever feels easier.
  • Understands tone and urgency — detects pain, anxiety, or worsening symptoms in real time.
  • Keeps concise summaries — turns each interaction into a chart-ready note for clinicians.
  • Flags yellow/red issues — escalates anything worrying and routes true emergencies to the on-call team or 911.
  • Reminds and encourages — medication prompts, wound-care tips, appointment nudges, all delivered with empathy.

How we built it

  • Patient interface: Twilio Programmable Voice & Messaging for frictionless calls and texts.
  • LLM stack: OpenAI o4 mini-high via a custom Model Context Protocol (MCP) that splits work among agents:
    • ToneAnalyzer – scores distress or confusion
    • FollowUpFlow – selects the next question or reassurance step
    • SummaryWriter – drafts a note in plain language or SOAP format
    • EscalationGuard – applies clinical red-flag rules
  • Speech tech: Audio Framing TTS for warm, natural delivery.
  • Backend: FastAPI on a lightweight VPS with PostgreSQL for audit-grade logging and webhook integrations to the EMR.
  • Security: End-to-end TLS, encrypted PHI at rest, and role-based access for staff dashboards.

Challenges we ran into

  • Latency: Balancing real-time voice with summarization and flag checks.
  • Nuance detection: Teaching the model to hear subtle worry versus casual chatter.
  • Escalation logic: Avoiding false alarms while never missing a real problem.
  • Tone consistency: Keeping the AI empathetic, not robotic, across both voice and SMS.
  • EMR mapping: Translating free-form patient stories into structured fields clinicians actually use.

Accomplishments that we're proud of

  • A fully working cross-channel assistant live-tested with mock patients in under 48 hours.
  • Instant one-click handoff: clinicians receive a digest plus the call recording when escalation triggers.
  • Modular MCP design—easy to swap models or add languages without touching core logic.
  • Achieved a sub-two-second average response in live voice loops, keeping conversations fluid.

What we learned

  • Follow-up isn’t just symptom checks; reassurance is half the job.
  • Patients mix voice, text, and images—meeting them on any channel builds trust.
  • Splitting LLM roles (collection, analysis, summary) makes debugging and compliance far simpler.
  • Empathy can be templated—but only if your prompts are written like a real nurse, not a bot.

What's next for Phi Medical

  • HIPAA & PIPEDA certification for pilot deployments.
  • Multi-lingual models (starting with French and Spanish) to widen access.
  • Rich media: let patients send wound photos that route to a computer-vision check before hitting the nurse’s inbox.
  • Analytics dashboard: track recovery trends, common concerns, and staffing impact.
  • EHR plugins: push summaries directly into Epic and Cerner so nothing gets copy-pasted.

We see Phi Medical becoming the always-on recovery companion that patients trust and clinicians rely on—closing the gap between discharge and the next appointment.

Built With

Share this project:

Updates