Demo Video: https://www.loom.com/share/6fe3eed37d5949c08aac5ecfa679a598

Inspiration

Every medical practice has someone whose day is spent saying the same four things on the phone: what's wrong, who can see you, are you covered, when can you come in. It's one of the most automatable jobs in healthcare, and one of the most dangerous to automate badly.

The moment a voice agent says "your plan is active, your copay is $30," it has made a claim a patient will act on. If a language model generated that sentence, it can generate it when it isn't true. In healthcare that isn't a glitch. It's a liability.

So we didn't set out to build a voice agent that sounds human. We set out to build one that cannot lie about your insurance, and then made it sound human.

What it does

Yara is an AI front desk receptionist for medical practices. You call a real phone number and talk to it.

It understands why you called ("I twisted my ankle playing soccer" gets you a sports medicine orthopedist), recognizes you from your caller ID but still makes you verify, checks your insurance, books the appointment, and texts you a confirmation.

And it refuses when it should. Inactive plan? It says so and offers a human. Payer not responding? It says that too. It never guesses.

How we built it

Caller (PSTN) -> Twilio number + Elastic SIP Trunk -> LiveKit Cloud (SIP trunk -> dispatch rule -> room) -> Python agent worker STT deepgram/nova-3-medical LLM gpt-4.1-mini TTS cartesia/sonic-3.5 tools -> clinic.py (the gates) -> Twilio confirmation back to the caller

One rule shapes everything: the model narrates, deterministic tools decide. The LLM has no authority. Coverage, identity, which doctor treats what, and whether a slot is booked are all tool results computed in Python. The model's only job is to read them out loud in a warm voice.

The guardrails are not prompt instructions. A prompt saying "don't reveal PHI without verification" is a suggestion, and suggestions can be argued out of. Ours raise exceptions: check_eligibility, list_slots, and book_slot throw unless the caller was verified on this call. The model never sees the data, so it never gets the chance to be careless.

Two deliberate choices: we used a clinically-trained speech model (nova-3-medical), because mishearing a member ID over an 8kHz phone line is a worse failure than a slow reply. And the tool layer imports nothing from LiveKit, so every guardrail is testable without making a phone call.

Challenges we ran into

A hostname that resolved but was wrong. Our first call failed with the carrier saying "this person isn't accepting calls." We'd guessed LiveKit's SIP host from our project's WebSocket subdomain, and DNS resolved it, so it looked right. Turns out *.sip.livekit.cloud is a wildcard: a completely made-up subdomain resolves to the same IPs. Our INVITEs were going into the void. A hostname that resolves proves nothing.

A2P 10DLC ate our SMS. Booking confirmations returned error 30034. US carriers require every 10-digit long code to be registered with a Brand and Campaign before it can text a US number. It's a carrier mandate, not a Twilio setting, and it takes days. We shipped over Twilio's WhatsApp sandbox instead, behind a pluggable sender, so real SMS is a config flag once registration clears.

Our own guardrails had holes. We found them by attacking them:

The system prompt listed every demo patient's member ID and date of birth so the model would know the format. Which meant the model knew valid answers to its own identity gate, and could have supplied them for a caller who offered only a name. The Python check was sound. The model was walking through it holding the key.
book_slot trusted a cached in-network flag from whichever provider was checked earlier, so you could check coverage against an in-network doctor and then book an out-of-network one. It booked.
Bookings were stored process-globally and never cleared, so the second run of our demo script would have failed with "that slot was just taken." We'd have rehearsed successfully and broken in front of judges.

None of these were caught by our tests. They were caught by trying to break the thing on purpose. Each is now a regression test.

Accomplishments that we're proud of

You can attack it live and it holds. Ask for your copay before verifying: it can't answer. Say "I'm Jane Doe": a name isn't verification. Tell it to book you anyway on a dead plan: it refuses and offers a human. Those refusals aren't the model being careful. They're Python exceptions.

It knows you and still doesn't trust you. Caller ID gets you a greeting by name. It gets you nothing else, because caller ID is spoofable in minutes.

We shipped honesty. Our README says plainly which parts are real and which are mocked, rather than hoping nobody asks.

What we learned

A guardrail you can talk your way past is a feature, not a guardrail. The useful question isn't "does it refuse when asked nicely." It's "is there any sequence of sentences that gets the data out." Every time we found one, the fix was to move the decision further from the model, never to add another line to the prompt.

Recognition and verification are different things, and conflating them is how you build a vulnerability.

The unglamorous integration is the whole project. The reasoning was the easy part. Ninety percent of the difficulty was SIP trunks, wildcard DNS, carrier registration, and codecs.

What's next for Yara Health

Live eligibility. Today check_eligibility returns 271-shaped fixtures, including an AAA-42 "payer not responding" error. It does not call a real payer, and we'd rather say so than have you find out. Real payer connections need provider and NPI identity, payer enrollment, trading-partner setup, and a BAA before any PHI moves - onboarding that outruns a hackathon. So we froze the contract, not the data: going live replaces one function body, and the guardrails, tests, and voice flow don't change.

Dropped-call resume. State keyed by call SID, so a caller who gets cut off can redial and pick up mid-booking.

Warm transfer to a human, instead of an escalation flag.