ClinicDesk

The AI front desk for Indian clinics: patients call or chat in English, Hindi or Kannada and get booked safely. Nemotron understands; a workflow decides.

Comment

Inspiration

A small clinic's front desk is one person doing three jobs: answering a phone that never stops, running the token queue, and collecting fees. Patients call in English, Hindi or Kannada. Missed calls are missed patients, and a rushed answer to "I have chest pain" can be dangerous. We wanted an AI front desk a clinic could actually trust.

What it does

  • Books by voice or chat in English, हिन्दी and ಕನ್ನಡ: finds real free slots, registers new patients with consent, confirms, and sends a payment link. Spoken numbers ("nine eight four five…", "दो नंबर") and spoken yes/no ("haan ji") work.
  • Catches emergencies first, in three languages, even if the AI model is down: the patient sees 108/112 (Tele-MANAS 14416 for self-harm) and staff get an alert with a chime.
  • Never gives medical advice, refuses prompt injection, never reveals other patients' data, blocks card numbers and OTPs.
  • Runs the token queue with live waits, and timed slots for doctors who use them.
  • Staff dashboard with roles (owner, receptionist, doctor), a live schedule, and the Agent X-ray: the real LangGraph graph lighting up node by node with every Nemotron call, guardrail decision and its cost.
  • Any AI agent can book over the open A2A protocol: the clinic publishes an Agent Card with five skills; a patient's own Nemotron Nano companion books for them and still asks the human to confirm.
  • Clinic onboarding with Tavily: the owner pastes the clinic's website; Tavily reads it, Nemotron drafts the listing, and every value is marked "on the page" or "check this" for review.

How we built it

The AI understands. The workflow decides. Nemotron 3.5 Lightning (on Nebius Token Factory) extracts what the patient said as JSON. A deterministic LangGraph booking flow does the rest: real slots from the calendar, phone validation, consent once, replies from templates. Nothing is booked until the patient says yes, enforced by LangGraph interrupts and checkpointed in Postgres so conversations survive restarts.

  • Models: Nemotron Lightning (understanding), Super (other replies, onboarding drafts), Ultra (staff handoff notes), Nano (the A2A companion), all through one OpenAI-compatible Token Factory API, with a cost ledger and a daily spend cap.
  • NVIDIA NeMo Guardrails as graph nodes: input rails (emergency, injection, payment secrets, privacy, medical) and output rails (no advice, no phone numbers, no times or fees that aren't in the tool results).
  • Real-time: one event stream feeds AG-UI over SSE (patient app), a WebSocket (staff dashboard, filtered by role), A2A status updates and OpenTelemetry traces.
  • Security: staff accounts with scrypt hashes, hashed revocable sessions, lockout, role-based access scoped to the clinic; patients own their conversations through a secret key; one-click demo roles for judges.
  • Voice: browser speech recognition and synthesis behind a swappable interface; the server rewrites every reply for the ear ("कल, सुबह 9:30", "500 ರೂಪಾಯಿ").
  • Quality: 488 backend tests on SQLite and Postgres, a 61-case red-team suite in CI plus a nightly live run, and a dev log of every problem we hit.

Challenges we ran into

Our first live booking told a patient "successfully booked" when nothing was booked. Letting the model drive bookings produced invented slots, ignored an 11-digit phone number and looped on "3". We rebuilt bookings as a deterministic flow where the model only extracts, which fixed all of it. Other lessons: language detection flipping between Hindi and romanised Kannada, "first visit" being read as "option 1", a too-short database column silently dropping cost records, Neon closing idle database connections mid-demo, and making voice understand "दो नंबर".

Accomplishments that we're proud of

Emergencies are caught in three languages with no model at all. Every one of the 61 red-team conversations passes in CI. Staff can watch exactly what the agent did, and what it cost. And a second AI agent booked an appointment end to end over A2A, asking its human before every decision.

What we learned

Put the model where it is strong (understanding messy multilingual text) and keep decisions in code. Safety features are only real when tested: the red-team suite caught gaps we would never have found by hand.

What's next

Real phone numbers and WhatsApp, appointment reminders and no-show follow-ups, a server-side Indian-language voice for Kannada, calendar and patient-record integrations, and pilots with clinics in Bengaluru.

Built With

  • a2a
  • ag-ui
  • fastapi
  • langgraph
  • nebius-token-factory
  • nemo-guardrails
  • nvidia-nemotron
  • opentelemetry
  • postgresql
  • python
  • react
  • render
  • tailwindcss
  • tavily
  • typescript
  • web-speech-api
Share this project:

Updates

Submission history