SwarMed AI

• 

• 

Inspiration

In crowded emergency rooms and rural Primary Health Centers (PHCs) across India, we witnessed two heartbreaking realities: doctors overwhelmed by manual triage, causing life-threatening delays for critical patients, and terrified families unable to understand complex, jargon-heavy lab reports written in English. We built SwarMed AI to bridge this gap—giving overworked healthcare workers an intelligent co-pilot while empowering patient families with clear, spoken insights in their native languages.

What it does

SwarMed AI is a dual-interface emergency intelligence platform:

For Doctors & ER Staff: It automates severity scoring (Red / Yellow / Green) using real-time patient vitals (SpO2, heart rate, BP, symptoms) and generates instant differential diagnosis suggestions to speed up critical care.

For Patient Families: It uses multimodal AI to scan complex lab report images, breaking down abnormal values into plain, non-alarmist explanations translated and spoken aloud in regional Indian languages (Hindi, Tamil, Telugu, Marathi, English).

For Rural Infrastructure: It features an edge-fallback mode that maintains basic triage capabilities even when internet connectivity drops in remote clinics.

How we built it

Frontend: Built with React and Tailwind CSS for a high-visibility, reactive dashboard designed for high-stress ER environments.

Backend Gateway: Python FastAPI handling asynchronous API requests and Pydantic data validation.

AI Intelligence Engine: Google Gemini 2.5 API using gemini-2.5-flash for high-speed structured JSON triage responses, and Gemini Multimodal Vision for direct OCR and semantic interpretation of medical report images.

Voice Synthesis: HTML5 Native Web Speech API for zero-latency, zero-cost regional language audio readouts (hi-IN, ta-IN, te-IN, mr-IN, en-IN).

Challenges we ran into

Medical Guardrails & Hallucinations: Ensuring the AI never assigned a safe "Green" status to a critically unstable patient. We solved this by implementing deterministic Python override rules (e.g., hard-coded triggers forcing a RED status if SpO2 drops below 90%, regardless of LLM output).

OCR Precision on Blurry Reports: Handwritten or low-quality clinic report scans caused extraction errors. We resolved this by engineering dual-pass verification prompts that cross-reference observed metric values against standard reference ranges.

Tone Sensitivity: Translating medical reports into local languages without triggering undue panic required strict prompt constraints focused on neutral, empathetic phrasing rather than alarmist terminology.

Accomplishments that we're proud of

Engineering an end-to-end multimodal pipeline that takes a raw lab report image and delivers a plain-language spoken audio explanation in regional languages in under 3 seconds.

Implementing a strict, clinical-grade JSON schema fallback that guarantees emergency triage alerts are rendered reliably without API output formatting failures.

Designing a zero-retention, privacy-first data handling architecture that respects patient confidentiality by processing images ephemeral in-memory.

What we learned

How to combine deterministic software logic with probabilistic LLM outputs to build safety-critical medical decision-support systems.

Deepened our understanding of web accessibility, particularly the power of browser-native Web Speech APIs for bridging digital literacy barriers in healthcare.

The nuances of prompt engineering for multimodal vision models when parsing complex medical tabular data.

What's next for SwarMed AI

ABDM / FHIR Integration: Linking SwarMed AI directly with India’s Ayushman Bharat Digital Mission health stack for automated patient history fetching.

Local Quantized Models: Deploying fully offline ONNX quantized vision models on low-cost single-board computers (like Raspberry Pi) for zero-internet rural PHCs.

Multi-Modal Vital Wearable Sync: Connecting real-time Bluetooth pulse oximeters and ECG monitors directly into the ER triage stream.

Built With

• fastapi
• frontend:-react
• html5-web-speech-api-backend:-python
• json
• rest-apis
• schema
• tailwind-css