Twilla

Healthcare workflows, from hello to done

Comment

The Story of Twilla: Reimagining Healthcare Access

This document details the inspiration, architecture, challenges, accomplishments, and future roadmap of Twilla—our conversational healthcare AI agent.

Inspiration

Every day, millions of patients dial their clinic, only to hear: "Your call is important to us, please continue to hold."

Statistics show that over 30% of patient calls go unanswered due to administrative overload. In healthcare, a missed call is not just a lost sale; it represents a delayed diagnosis, a missed physical therapy session, or a family caregiver left in the dark.

Mathematically, the probability of a caller dropping off the queue increases exponentially with time $t$:

$$P(\text{drop}) = 1 - e^{-\lambda t}$$

where $\lambda > 0$ represents the user's fatigue coefficient. By deploying an AI agent that picks up the call instantly ($t \to 0$), the drop-off probability collapses to zero:

$$\lim_{t \to 0} P(\text{drop}) = 0$$

We built Twilla to solve this bottleneck. We wanted to create an agent that is empathetic, grounded in clinic policies, capable of booking schedules, and robust enough to handle high-risk emergencies securely.

What it does

Twilla is an end-to-end clinical assistant operating over phone lines and SMS channels.

  • Patient Intake: Collects basic demographics, date of birth, and reasons for visits from new patients, registering them directly into the clinic's database.
  • Intelligent Scheduling: Queries open slots in real-time, books appointments, and sends instant SMS confirmations.
  • Insurance Check: Matches patients' plans against clinical procedure codes to calculate copays and deductible responsibilities.
  • Proactive Reminders: Processes schedules in the background to send SMS and outbound call reminders.
  • Caregiver Loop & Escalation: Reaches out to caregivers after appointments to verify medication adherence and recovery. If symptoms are deteriorating, it flags a clinical nurse escalation.

How we built it

Twilla's architecture is modular and built on a Node.js/Express framework:

                  ┌───────────────┐
                  │ Patient Call  │
                  └───────┬───────┘
                          │ (Telephony)
                  ┌───────▼───────┐
                  │    Twilio     │
                  └───────┬───────┘
                          │ (Webhooks via Serveo)
                  ┌───────▼───────┐
                  │  Express App  │
                  └───────┬───────┘
         ┌────────────────┼────────────────┐
 ┌───────▼───────┐ ┌──────▼──────┐ ┌──────▼──────┐
 │  Guardrails   │ │   GPT-4o    │ │ SQLite DB   │
 │ • Emergency   │ │   Agent     │ │ (sql.js)    │
 │ • Clinical    │ │ • 10 Tools  │ └─────────────┘
 └───────────────┘ └──────┬──────┘
                          │ (Context Search)
                   ┌──────▼──────┐
                   │  Knowledge  │
                   │    Base     │
                   └─────────────┘
  • Telephony: We used Twilio Voice and SMS to handle telephony. Rather than running complex raw WebSockets, we used Twilio's enhanced speech recognition <Gather> loop for maximum reliability.
  • AI Orchestration: Powered by GPT-4o with function calling. The agent uses 10 custom Javascript tools to interact with database states deterministically.
  • Grounding Context: A vector-style search grounds the AI's responses using static JSON databases for clinic FAQs, insurance plans, and prep procedures. The similarity score $S$ between a query $q$ and doc $d$ ensures factual responses:

$$S(q, d) = \frac{\mathbf{q} \cdot \mathbf{d}}{|\mathbf{q}| |\mathbf{d}|}$$

  • Database: Swapped native compiled drivers for pure JS sql.js (SQLite engine) to prevent environment native compilation bottlenecks on standard systems.
  • Admin Console: Built a premium, Twilio-branded light-mode dashboard that displays live metrics, active schedules, and a live message transcript panel.

Challenges we ran into

  1. Native Node Compiler Failures: We initially set up better-sqlite3 for SQL persistence. However, compilation of C++ native binaries crashed due to missing Visual Studio compiler tools. We spent hours refactoring our database wrapper to run on sql.js, a pure WebAssembly/JS SQLite port.
  2. OpenAI Quota Limits (Rate Limit 429): During testing, we hit OpenAI's spending/quota limits. To prevent our live telephone calls from crashing during judge evaluations, we engineered a local Clinical Fallback Engine that detects intents using keywords and executes the exact same DB workflows (intake, scheduling, guardrails) offline.
  3. Public Tunnel Warning Barriers: Localtunnel and Ngrok recently introduced interstitial warning pages for free accounts. Twilio webhooks are automated and cannot bypass these pages, leading to "Application Errors." We resolved this by integrating a self-healing Serveo SSH tunnel monitor that runs in the background and bypasses warning pages for JSON/XML requests.

Accomplishments that we're proud of

  • Interactive Simulator: The admin console includes an interactive chat simulator allowing judges to interact with the AI agent without calling international numbers.
  • Bulletproof Guardrails: Emergency detection is fully hardcoded to override the AI and route patients to 911 immediately if high-risk symptoms (e.g. chest pain) are reported.
  • PII Redaction: Built automatic regex scrubbers that sanitize database logs, replacing SSNs, DOBs, and phone numbers with tokens:

$$\text{Plaintext Logs} \xrightarrow{\text{Scrubber}} \text{Sanitized Audit Logs}$$

What we learned

We learned that building conversational agents for healthcare requires a shift in priority from creative intelligence to strict reliability.

We cannot allow LLMs to freely generate answers about clinic hours or procedures. Grounding models via deterministic tool execution is mandatory. We also realized that developers need simple visual monitoring; our admin console made debugging conversational trees 10x faster than reading command-line logs.

What's next for Twilla

  1. FHIR & EHR Integration: Connecting Twilla directly to HL7/FHIR APIs (like Epic and Cerner) to sync real-time patient charts.
  2. WebSocket Audio Streaming: Migrating Twilio Gather to raw Media Stream WebSockets to support interruption handling (allowing patients to talk over the agent).
  3. Multi-lingual support: Adding localized clinical vocabularies in Spanish, Hindi, and Mandarin.

Built With

Share this project:

Updates

Submission history