Clover

Find the specialist of specialists for your symptoms, and let’s work together to put an end to the medical merry-go-round and ensure that complex conditions receive the care they deserve.

Comment

Clover - Intelligent Practitioner Matching

AI-powered care navigation system that matches patients to the right specialist using multimodal input, semantic search, and a reverse triage heuristic — live at front-app.leo-sandbox.workers.dev.

Team

Name LinkedIn Profile Link
Louis Brulé Naudet linkedin.com/in/louisbrulenaudet
Leo Arsenin linkedin.com/in/leoarsenin

The Problem

Finding the right specialist is one of the most overlooked friction points in the patient care journey. Most booking systems optimize purely for the earliest available slot, regardless of whether that practitioner is a good clinical fit for the patient's specific condition. This mismatch leads to wasted consultations, unnecessary referral chains, and delayed care - particularly for patients with complex or ambiguous symptoms who struggle to self-identify the correct specialty.

What It Does

The system is an AI-assisted medical appointment facilitation platform designed to reduce friction in the patient care pathway. It operates through a four-stage pipeline:

Stage 1 — Vocal Data Ingestion: Patients interact via a conversational interface. The audio is processed by Mistral's real-time speech-to-text model, which normalizes the heterogeneous inputs into a unified structured patient profile validated through Zod schemas.

Stage 2 — Patient Profiling and Risk Stratification: The system evaluates the nature of the complaint and contextual data to produce a structured metadata payload: clinical tags, urgency scores, specialization requirements, and constraint parameters (geography, availability windows). Every object in the data model is represented as a Zod schema for systematic validation.

Stage 3 — Filtered Semantic Search and Practitioner Matching: A hybrid vector search runs simultaneously over two data categories — practitioner profiles (specialty, availability, scheduling) and qualitative patient feedback (consult motivation and context, sourced from the Alan insurer platform as a trusted private channel). Rather than surfacing the earliest available slot, the system applies a reverse triage heuristic: candidates are ranked by clinical relevance to the patient's specific condition, tolerating longer wait times in exchange for higher specialization accuracy when the assessed risk level permits.

Architecture (how the stack connects)

The diagram below shows the real-time path from the client through the Worker API Durable Object to RAG, Mistral, optional Workers AI, ElevenLabs, ChromaDB, and Nebius.

flowchart LR
  subgraph clients [Clients]
    Laptop["💻 Laptop / Browser"]
    Mobile["📱 Mobile"]
  end

  subgraph worker [Cloudflare Worker API :8725]
    DO["Durable Object\nTranscriptionRoom\n\n• WebSocket session mgmt\n• PCM → transcript pipeline\n• Tool calling (RAG dispatch)\n• Conversation history\n• ElevenLabs TTS\n• Stream MP3 back to client"]
  end

  subgraph external [External Services]
    subgraph rag [RAG API — worker-rag :8726]
      RAGWorker["ChromaDB vector store\nMistral Embed\nsearch_patient_rag tool\n/search /add endpoints"]
    end
    Mistral["Mistral AI\n\nVoxtral — realtime STT (WS)\nmistral-medium — chat LLM\nmistral-embed — RAG"]
    CFAI["Cloudflare Workers AI\n\nGLM 4.7 Flash\n(alt chat model, no tools)"]
    EL["ElevenLabs TTS\n\neleven_turbo_v2_5\nWS stream → MP3 chunks"]
    ChromaDB["ChromaDB Cloud\n\nVector store\nEmbeddings search"]
    Nebius["Nebius Token Factory\n\nAlt LLM inference\nAPI key management"]
  end

  Laptop -->|"WebSocket\n🎤 PCM audio stream"| DO
  Mobile -->|"WebSocket\n🎤 PCM audio stream"| DO

  DO -->|"HTTP / Queue"| RAGWorker
  DO -->|"STT + Chat API"| Mistral
  DO -->|"AI Binding"| CFAI
  DO -->|"Text → TTS (WebSocket)"| EL

  RAGWorker --> ChromaDB
  RAGWorker --> Nebius

  EL -.->|"🔈 MP3 audio streamed back"| Laptop
  EL -.->|"🔈 MP3 audio streamed back"| Mobile

Tech Stack

Leave a 🚀 next to any partner technologies used (e.g., Nebius 🚀)

Technology Role
Mistral AI 🚀 Voxtral — realtime STT over WebSocket; mistral-medium — chat LLM with tool calling (RAG dispatch); mistral-embed — embeddings for Chroma search
Nebius 🚀 Token Factory and alternate LLM inference / API key management; used from worker-rag alongside Chroma
ElevenLabs 🚀 eleven_turbo_v2_5 — TTS over WebSocket; MP3 chunks streamed back to the client
ChromaDB Cloud Vector store for practitioner and review records; worker-rag exposes /search, /add, and search_patient_rag tooling
Cloudflare Workers AI GLM 4.7 Flash — optional alt chat model (Workers AI binding; no tools path)
Durable Objects (TranscriptionRoom) WebSocket session management, PCM → transcript pipeline, conversation history, orchestration to Mistral / RAG / ElevenLabs
Hono HTTP API on worker-api (:8725); worker-rag (:8726) for RAG endpoints
React + Vite + Tailwind CSS v4 Frontend SPA (live demo)
Zod Systematic validation of all data model objects (patient profiles, risk payloads, practitioner records)
Cloudflare Workers + Turborepo Monorepo deployment infrastructure

Special Track

  • [ ] Alan Play: Living Avatars
  • [ ] Alan Play: Mo Studios
  • [ ] Alan Play: Personalized Wrapped
  • [ ] Alan Play: Health App in a Prompt
  • [ ] Alan Precision

General track submission. Partner tracks: ElevenLabs, Mistral AI, Nebius.

What We'd Do Next

If we had more time, we would:

  • Polish the frontend further to make the patient-facing experience more intuitive and accessible.
  • Build a scheduling system with reminders and push notifications: when a specialist posts new availability slots, patients who matched with that practitioner would be notified automatically — surfaced directly within the Alan application.
  • Extend the monitoring module with richer adaptive rerouting logic, including longitudinal tracking of condition evolution across multiple follow-up submissions.
  • Explore deeper integration with Alan's internal data pipelines to improve the quality and freshness of practitioner and review embeddings in real time.

Built With

Share this project:

Updates