PulseCheck

Before vs. AI Triage Transformation: Replacing waiting room bottlenecks and manual documentation with autonomous triage.
Multilingual Mobile Intake: Patients choose from 5 languages with instant voice or text support.
Voice-First Scribe: Native speech transcription with clinical semantic input validation.
Deterministic Triage: Objective AI-locked severity rating and on-duty physician routing.
AI Clinical Summary & Review: Pre-consultation summary featuring objective pain scoring, clinical bullet points, and physician assignment.
Digital OPD Queuing: Automated daily-refreshing tokens and mobile WhatsApp/SMS dispatches.
Authorized Physician Onboarding: Secure provider registration guarded by hospital master key validation and department assignment.
Provider Authentication Portal: Role-based login providing secure physician access to personalized OPD queues and EHR charts.
Physician EHR Dashboard: Structured SOAP notes ready for 1-click hospital EHR export.

Inspiration

Emergency department (ED) crowding and clinical documentation overload represent dual crises in modern healthcare. Clinicians spend significant portions of their clinical shifts navigating complex electronic health record (EHR) systems instead of focusing on direct bedside care. At the same time, emergency waiting bays are bottlenecks: non-English speaking patients face language barriers, non-clinical desk staff lack diagnostic acuity, and sudden physiological decompensations risk being overlooked in crowded waiting areas.

We designed PulseCheck to solve the front door of clinical intake. By shifting intake to the patient’s own smartphone via a simple waiting room QR code, patients can articulate their medical complaints in their native language using natural voice or text. Our clinical engine then bridges the communication gap by translating, structuring, triaging, and synthesizing a comprehensive SOAP/HPI note directly for the physician before the patient steps into the examination room.

What It Does

PulseCheck is an AI-driven multilingual patient triage and autonomous clinical scribe system operating across two synchronized interfaces:

Mobile-First Patient Intake: A zero-install web app accessed via QR code offering full voice and text intake across five languages (English, Spanish, Hindi, French, Mandarin).
Clinical Integrity & Anti-Gibberish Guard: Semantic validation filters meaningless keystrokes, rejecting non-medical entries before running clinical extraction.
Deterministic Acuity Triaging: Clinical LLM pipelines extract chief complaints, compute duration timelines, and lock objective triage severity scores on a scale of $S \in [1, 10]$ based on established Emergency Severity Index (ESI) principles:

$$S = f(\text{Acuity}, \text{Pain}, \text{Vitals}, \text{RedFlags})$$

Physician Scheduling & Dynamic Routing: Patients select consulting physicians filtered strictly by live availability status (🟢 On Duty vs 🔴 Off Duty).
Digital OPD WhatsApp Tokens: Generates sequential daily tokens (e.g., #T-003) dispatched directly to the patient's messaging app with real-time wait estimations.
Active Code-Red SOS Dispatch: A one-touch emergency bypass triggers an audible chime and persistent visual alert banner across all authenticated doctor dashboards while generating a top-priority #EMERGENCY-SOS scribe ticket.
Role-Guarded Provider Portal: Doctors authenticate via Hospital Admin Keys (HOSP-PULSE-2026) to access isolated departmental patient queues, review raw audio with original transcripts, and export structured SOAP notes with one click.

How We Built It

PulseCheck was engineered using a modern full-stack web and artificial intelligence architecture:

Frontend Framework: React 18, TypeScript, and Vite with Tailwind CSS for high-contrast, accessible UI design.
Audio & Speech Pipeline: Standardized with the Web Speech API (SpeechRecognition / webkitSpeechRecognition) for real-time multilingual streaming, alongside the HTML5 MediaRecorder API to capture clean audio payloads across mobile platforms.
AI Intelligence & Multimodal Parsing: Powered by Gemini 1.5 Pro and Gemini 1.5 Flash via structured JSON schemas, running clinical entity extraction, translation, timeline modeling, and SOAP generation.
Offline Fallback Engine: Deterministic regex heuristic algorithms that handle parsing if network connectivity drops or API quotas exhaust.
Cross-Tab Synchronization: Inter-tab storage event listeners and state synchronization managing real-time emergency dispatch and provider queue updates across multiple devices.

Challenges We Faced

Mobile Audio Hardware Contention: On mobile browsers, initiating getUserMedia for recording while running webkitSpeechRecognition locked the underlying audio tracks, resulting in premature no-speech timeout errors. We resolved this by unifying stream handling, setting continuous = false, interimResults = true, and adding a startup delay to allow hardware warm-up.
Input Fuzzing and Nonsense Detection: Free-form text fields invite accidental keystroke spam (e.g., "asdasdx"). Building a heuristic vowel-to-consonant validation check coupled with an LLM clinical validation prompt ensured that non-medical noise never triggers an erroneous triage score.
Cross-Device Audio Container Alignment: Handling variable audio containers (audio/webm, audio/mp4, audio/aac) between mobile Safari and desktop Chrome required base64 blob normalization to ensure multi-modal LLM decoders process audio without failure.

What We Learned

Building PulseCheck reinforced that healthcare AI is only as good as its safety boundaries. We learned how to build bounded schemas where machine learning models act strictly as structured extraction engines rather than ungrounded diagnostic authorities. Balancing clinical rigor with patient usability—especially under high-stress emergency conditions—demands intuitive UX, clear feedback loops, and resilient fail-safes at every step of the workflow.