CareLine AI

A multimodal patient intake & care-coordination agent that turns the slowest step in healthcare — getting a patient into the system. Collapses 70 minute of work to 90 seconds.

Comment

Inspiration

Today's healthcare intake is stuck in the 1990s: patients discharge from hospitals and wait 3–7 days to start home care because someone has to manually call them back, collect their information, verify insurance, and book a visit. That someone is a coordinator spending 70 minutes per patient on phone tag, faxes, and spreadsheets.

We watched patients readmitted because they couldn't reach their care team in time. We saw coordinators buried. We asked: what if a patient could say "I need home care" once — voice, text, any language — and walk out with a booked nurse visit 90 seconds later?

## What it does

CareLine AI is a multimodal patient intake agent that closes the loop: referral → intake → verification → appointment, all in one conversation.

A patient (or family member) texts, voice-calls, or taps a link. They speak or type in English, Hindi, Spanish, or any language Deepgram recognizes. Cara (our AI coordinator) collects what she needs—name, diagnosis, insurance, home address—from text, voice notes, photos of insurance cards, and PDF referrals in the same conversation. She asks clarifying questions like a human coordinator, with warmth and empathy. Once intake is confirmed, she instantly books them with a real clinic calendar (Cal.com API), not "we'll call you later." The patient walks away with an appointment, a reference ID, and a booked nurse visit on Monday at 9 AM.

The clinic coordinator sees everything live on a dashboard: sessions filling in real-time, guardrail events (when Cara refuses medical advice), eligibility verified in 0.2 seconds, and booked appointments synced to their real calendar. No data entry. No callbacks. No bottleneck.

## How we built it

Frontend: React dashboard polling Cloudflare Workers API every 2 seconds (shows sessions, transcript, events, booked appointments, SOC packet JSON). Tap-to-answer WebRTC call page (no Twilio number needed yet).

Voice Agent: Twilio ConversationRelay (real-time WebSocket) + Deepgram nova-3 multilingual STT (auto-detects language mid-call) + ElevenLabs TTS (Jessica voice, native American English). When patients talk over Cara, we queue their speech (never drop it). When they send docs via WhatsApp mid-call, she acknowledges it live on the call within 3 seconds.

Brain: Groq llama-3.3-70b LLM (primary) with a 6-model resilience chain (Groq GPT-OSS-120b, Groq Scout, Workers AI llama-3.3-70b-fp8-fast, OpenRouter free models). JSON schema extraction for zero-hallucination field capture. Deepgram + ElevenLabs handle 5+ languages with correct feminine grammar (Hindi verb forms, Spanish/French pronouns, etc.) — Cara is a woman, and language respects that.

Calendar: Cal.com v2 API (real clinic availability, real bookings). Fallback to D1 SQLite local calendar. Appointment reschedule via text or voice using the same agent (no separate "rescheduling portal").

Eligibility: Instant automated check (mock payer rules; production uses EDI 270/271) — work that coordinators queue for "1 business day," done in 0.2s.

Database: Cloudflare D1 (SQLite) — sessions, messages, events, appointments, guardrails, knowledge chunks. Embeddings via Workers AI bge-base-en-v1.5, RAG retrieval to ground answers in policy (no invented coverage rules).

Infrastructure: Cloudflare Workers (serverless, zero cold start), D1 (free tier), Workers AI (free tier embeddings), all on $25 Twilio credit. No backend bill. No provisioning.

## Challenges we ran into

  1. Free-tier quota exhaustion: Groq's 100K token/day free quota ran out mid-testing. Solution: Built a 6-model fallback chain with independent quotas so one provider going down never breaks the conversation.

  2. Multilingual TTS was broken: We used Amelia (British accent), not American. Solution: Switched to Jessica (ElevenLabs, native American English, properly tuned for expressiveness).

  3. Feminine grammar in gendered languages: LLM was using masculine Hindi verb forms ("करूँगा"instead of "करूँगी").Solution: Explicitly told the prompt Cara is a woman,

  4. Feminine grammar in gendered languages: LLM was using masculine Hindi verb forms ("करूँगा"instead of "करूँगी").Solution: Explicitly told the prompt Cara is a woman, with feminine examples per language.

  5. Caller speech was being dropped when they talked during Cara's processing. Solution: Queue pending prompts instead of discarding; merge and process them in order.

  6. PDF extraction model retired: OpenRouter's Google Gemini free tier was pulled. Solution: Switched to Workers AI env.AI.toMarkdown() for scanned PDFs.

  7. Twilio KYC blocking everything: Account compliance profile unapproved → can't buy numbers or send outbound REST. Solution: Designed tap-to-answer WebRTC call page (patients tap a link Cara texts, WebRTC opens incoming-call screen, no PSTN number needed yet).

  8. Cal.com booking timing: Slot times were booked at 9 AM when the agent said 11 AM (timezone confusion). Solution: Slot IDs now derive from Unix timestamp mod 1000000 (stable across turns, can never mix up times), and Cal reschedule now persists the new booking ID so re-reschedules never hit dead UIDs.

## Accomplishments we're proud of

  • End-to-end in 90 seconds: PDF → fields → confirmation → real appointment, live on dashboard. Judges will watch a session transform from empty to "booked" in real time.
  • Zero hallucination: Deterministic SOC packet (built from typed fields, never LLM-generated), JSON schema validation on all decisions, RAG grounding so policy is never invented.
  • Truly multilingual: Deepgram auto-detect + language-tagged TTS means Hindi callers hear Jessica speaking Hindi natively, with correct feminine grammar. Not a translation; native fluency.
  • Cross-channel in one session: Text a voice note, switch to a call, send a photo via WhatsApp mid-call, pick an appointment time on voice—same session record, one unified intake. Judge sees it all in the transcript.
  • Real calendar integration: Not a demo calendar—Cal.com bookings sync to the clinic's real calendar. Reschedule via text and it moves the real booking.
  • No infrastructure bill: Cloudflare Workers + D1 free tier + Groq/OpenRouter free models + $25 Twilio credit = $0 per patient (at scale, minimal egress).

## What we learned

  • Fallback chains beat single providers: One quota exhaustion wouldn't have killed us if we'd built it on day 1.
  • Language isn't one-size-fits-all: Feminine grammar, native speakers, and cultural respect aren't add-ons—they're table stakes for a real product serving diverse patients.
  • Regulatory constraints are real: Twilio KYC isn't a blocker, it's a design constraint. Tap-to-answer WebRTC works today; outbound PSTN works tomorrow.
  • State machines prevent chaos: Explicit state (greeting → collecting → confirming → scheduling → complete) with guardrails (end_call only on goodbye) keeps the agent sane.
  • Synchronous webhook replies beat REST: Twilio's 12s webhook budget + synchronous TwiML response beats trying to send REST while processing. Works even before KYC.

## What's next for CareLine AI

  • Trust Hub KYC approval (user has gov't ID ready): Buy a Twilio number, enable outbound voice calls. "Cara calls the patient" instead of "patient taps to call Cara."
  • Real payer API: EDI 270/271 calls to insurance companies (seconds, not hours) so eligibility is 100% accurate and denials are caught before the visit.
  • Clinic dashboard actions: Coordinators can text patients, reschedule appointments, mark visits complete—all from the dashboard.
  • Voice agent field audits: When Cara asks for something, show the clinic coordinator the extracted field in real-time so they can flag misheards before booking.
  • Regional deployment: Spin up instances for other home health agencies on the same codebase (Workers for Platforms + multi-tenant D1).

Built With

Share this project:

Updates