HealthSwarm

A swarm of AI agents that find, call, and book the right doctor for you, in any language, before you finish your coffee

Comment

Inspiration

Booking a specialist takes 26 days on average in the US not because of care quality, but because coordination is manual, fragmented, and language-dependent. We automated the part nobody fixed: the phone calls.

What it does

One message. Six agents. A confirmed appointment.

You say "Book me a dermatologist near Koreatown." In parallel: your insurance and medical history are loaded, nearby clinics are searched geospatially, candidates are LLM-scored and ranked. The top 3 clinics are called simultaneously via conversational voice AI in the patient's language. Each call is transcribed, translated, and parsed for availability, insurance status, and wait time. A final LLM judge picks the best and returns it. Under 60 seconds, end to end.

How we built it

Six Fetch.ai uAgents on Agentverse, each with a permanent address and mailbox routing. No direct HTTP between agents all communication via ASI:One chat protocol.

  • swarm-profiler + swarm-finder run in parallel on every request MongoDB 2dsphere index handles geospatial clinic search
  • swarm-matcher scores candidates concurrently with ASI:One, then judges all call evidence to pick a winner
  • swarm-caller fires concurrent outbound calls via ElevenLabs Conversational AI + Twilio through a FastAPI voice gateway
  • swarm-fingerprint detects language, translates transcripts, extracts structured facts via LLM
  • Dashboard: Next.js + React Flow fed by a FastAPI SSE relay receiving live beacons from every agent

Challenges we ran into

  • Coordinating six parallel agents to fan out and converge cleanly race conditions in transcript polling took real debugging
  • ElevenLabs needs precise per-call context (name, language, insurance, specialty) injected every time getting this right across three languages was non-trivial
  • OSM clinic data has gaps: missing phone numbers, wrong categories, stale entries required filtering and fallback logic throughout

Accomplishments that we're proud of

  • Fully working end-to-end pipeline real calls, real transcripts, real LLM judgments, zero mocked critical paths
  • Three multilingual personas (Spanish, Korean, Hindi) tested end-to-end including mid-call language detection
  • Six agents running live on Agentverse with permanent addresses not local scripts
  • A dashboard driven by actual agent beacons, not a scripted animation

What we learned

  • Agents with persistent identity on Agentverse behave differently and better than stateless scripts
  • Parallelism isn't an optimization, it's the core product decision. Sequential agents would make this unusable.
  • Voice AI quality is entirely determined by the context you inject upfront
  • A pure-observer dashboard relay made the whole system easier to debug and far better to demo

What's next for HealthSwarm

  • EHR via FHIR/HL7- replace synthetic patient data with real medical records
  • Insurance pre-auth agent - negotiate authorization before the booking call is placed
  • More languages - Mandarin, Tagalog, Armenian (LA's top 5 languages are the immediate target)
  • Provider API integrations - skip the phone call entirely where Epic, Zocdoc, or Practo APIs exist

Built With

Share this project:

Updates