ScribeHealth

An AI consultant for Nigerian hospitals that transcribes code-switched local languages and English from patient audio, extracting structured SOAP clinical notes to assist doctors in fast decisions

Comment

Inspiration

Nigeria is a beautiful tapestry of over 250 ethnicities, but this diversity creates a critical bottleneck in healthcare. In emergency rooms across the country, valuable minutes are lost because distressed patients communicate their symptoms by "code-switching"—blending English with Pidgin, Yoruba, Igbo, or Hausa. Critical context gets lost in translation, especially with less literate patients. We asked ourselves: What if technology could bridge this linguistic divide instantly, turning raw, mixed-language spoken testimonies into structured, life-saving clinical insights? That question birthed ScribeHealth.

What it does

ScribeHealth is an AI-powered clinical consultant designed for the local Nigerian context. At the patient’s bedside or in the ER, a doctor or attendant simply presses record. The patient describes their ailments naturally, mixing languages however they feel most comfortable. ScribeHealth captures the audio, transcribes the complex code-switched multi-lingual dialogue, and interprets the clinical meaning. Within seconds, the engine filters the raw conversation and structures it into a standard SOAP (Subjective, Objective, Assessment, Plan) clinical note format. Doctors instantly get a clean, parsed summary to make rapid, accurate decisions.

How we built it

We engineered a modern, ultra-performant, full-stack application built to handle high concurrency and deep linguistic processing:

  • Frontend: A highly responsive user interface built using React, optimized with clean state management to handle real-time audio recording states and immediate visual streaming of clinical notes.
  • Backend: A robust, enterprise-grade architecture engineered with NestJS, utilizing its modular design to cleanly decouple our authentication, database triggers, and AI pipeline micro-services.
  • Database & Auth: Powered by Neon, a serverless Postgres database that allows us to scale storage compute instantly, and secured via Better Auth for seamless, developer-friendly, and highly secure multi-tenant sessions.
  • AI & Language Processing Layer: A powerful two-step orchestration pipeline:
  • We integrated Intron Health AI—the premier clinical speech recognition engine built specifically for African accents and localized mixed languages—to accurately transcribe the raw audio.
    1. The transcription payload is instantly routed to Groq, leveraging its lightning-fast LPU inference speeds to read the raw multi-lingual conversation, strip the noise, and extract structured JSON mapped to standard medical SOAP metrics.

Challenges we ran into

Building for multilingual nuance presented two core bottlenecks:

  1. The Infrastructure Latency Stack: Chaining an audio transcription engine (Intron Health) into a Large Language Model (Groq) can introduce cumulative API latency delays. If not managed carefully, doctors would wait too long in emergency scenarios. We tackled this in NestJS by optimizing response streams and ensuring our connection to Neon's serverless edge connection was warm and highly pooled.
  2. Context Bloating & Rate Limits: During long patient explanations, the raw transcribed text can blow past processing windows, triggering gateway drops or server busy exceptions. We built custom middleware in NestJS to clean, structure, and token-prune the incoming data stream right before it hit the Groq execution engine.

Accomplishments that we're proud of

  • Blazing Fast SOAP Extraction: By leveraging Groq's raw speed alongside Intron Health's deep localization, we achieved near-instantaneous transcription-to-SOAP generation that feels incredibly smooth.
  • Flawless Security Framework: Integrating Better Auth allowed us to ensure strict HIPAA-compliant data boundaries between different hospital attendants and patient profiles right out of the box.
  • Linguistic Intent Mapping: ScribeHealth does not just translate words; it translates intent. Seeing the system successfully parse a mixture of fragmented Pidgin and local dialects into a flawless, textbook-grade clinical SOAP analysis was deeply rewarding.

What we learned

We learned that specialized pipelines outperform generic monoliths. Instead of trying to make a single model do everything, separating the heavy lifting—using Intron Health for understanding complex African dialects and Groq for blazing-fast logical extraction—yielded superior accuracy and speed. We also learned to respect the mathematical bounds of system throughput. When routing data through consecutive API steps, the total transaction time $T_{\text{total}}$ is bounded by the network latency plus processing time: $$T_{\text{total}} = t_{\text{auth}} + t_{\text{intron}} + t_{\text{groq}} + t_{\text{neon}}$$ By decoupling database session state updates using asynchronous workers in NestJS, we minimized $t_{\text{neon}}$ to near zero, keeping our application runtime secure and fast.

What's next for ScribeHealth

We are expanding ScribeHealth from a smart parser into a comprehensive operational clinical suite:

  • Offline Audio Buffering: Allowing remote healthcare facilities with zero internet access to queue recordings locally in the React application state and batch-upload them the moment they hit minimal network grids.
  • EHR API Connectors: Designing NestJS webhooks that map our extracted SOAP JSON objects directly into popular open-source Electronic Health Record (EHR) systems used widely across West Africa.
  • Multi-Dialect Training Sets: Working to expand our contextual prompts to cover even more niche regional sub-dialects across different borders.

Built With

Share this project:

Updates

Submission history