MediLogue

An AI-powered Universal Dialect & Triage Engine that translates regional dialects and sign language into structured, professional clinical summaries for instant medical prioritization.

Comment

Inspiration

In rural and remote regions, access to quality healthcare is severely limited by an invisible yet devastating barrier: language and communication. Many elderly and indigenous patients can only articulate their medical emergencies in deep, localized native dialects. When they encounter medical practitioners in urban emergency rooms, critical symptoms are often misunderstood, leading to tragic misdiagnoses or fatal triage delays.

Furthermore, speech and hearing-impaired patients face an even steeper hurdle, lacking real-time tools to convey urgency. We built MediLogue to dismantle these communication walls and ensure that no patient is left unaccounted for simply because of how they speak or sign.

What it does

MediLogue functions as an intelligent, real-time Universal Dialect & Triage Engine tailored for emergency healthcare workflows. Operating through a high-performance console, the system processes incoming multi-modal data streams:

  • Vocal Node (Audio Input Stream): Captures raw phonetic audio spoken in heavy regional dialects and accurately interprets the underlying medical complaints.
  • Vision Link (Vision Sign Stream): Actively captures hand gestures, offering accessibility support by translating international and local sign languages.
  • Clinical Context & Biometrics: Integrates structured patient demography (Age, Gender, and Medical History) with physical indicators like Blood Pressure, Oxygen Saturation ($SpO_2$), and body temperature.
  • Agentic Triage Evaluation: Correlates the translated data against clinical benchmarks to immediately generate a prioritized clinical report, assigning risk multipliers to assist doctors in fast-tracking high-risk cases.

How we built it

MediLogue is engineered using a robust full-stack architecture optimized for low-latency operations:

  • Frontend UI: Built using React and TypeScript with a dark, high-density HUD console theme engineered with Tailwind CSS for distraction-free clinical navigation.
  • Backend API Gateway: Powered by Python and FastAPI, managing real-time data ingestion pipelines, phonetic audio buffering, and multi-modal stream routing.
  • Core Intelligence (The Brain): Orchestrated by Alibaba Cloud DashScope API, leveraging advanced Qwen Large Language Models to perform deep cultural semantic translations, clinical reasoning, and automated triage documentation.

Challenges we ran into

  • Nuance Mapping: Dialects do not have direct 1:1 dictionary translations. Mapping raw phonetic colloquial terms into structured medical terminology required extensive prompt calibration and agentic verification.
  • Dynamic Risk Multipliers: Designing an evaluation engine that balances subjective verbal complaints with objective numeric biometrics (e.g., matching a complaint of chest pain with an elevated body temperature of $39.2^\circ\text{C}$) required structured output validation via JSON schemas.
  • Deployment & KYC Hurdles: Navigating advanced regional registration and identity verification blocks during the deployment process forced us to refine a highly modular decoupled architecture—ensuring our system remains production-ready to scale across cloud infrastructures effortlessly.

Accomplishments that we're proud of

  • High-Density HUD UI: Successfully designing and implementing a futuristic, production-grade medical console interface that maps real-time AI outputs and biometric components beautifully without cluttering the clinical workflow.
  • Multi-Modal Synchronization: Successfully routing both audio-based dialect pipelines and vision-based sign language data structures into a singular, unified English clinical reporting format.
  • Contextual Medical Reasoning: Training the underlying LLM agents to accurately weigh critical patient demographic risks (like age and history) alongside raw symptoms to generate reliable triage justifications.

What we learned

Building MediLogue highlighted the true power of Large Language Models as reasoning engines rather than simple text generators. We learned how to structure agentic medical pipelines that safely handle sensitive demographic attributes (Age and Gender) to calculate triage priority accurately. More importantly, this journey solidified our belief that true accessibility in tech requires designing for the most vulnerable and isolated communities first.

What's next for MediLogue

  • Expanded Regional Models: Moving beyond initial prototype tracks to actively train and map broader indigenous and deep regional dialects across Southeast Asia.
  • Native Cloud Integration: Resolving regional infrastructure restrictions to fully deploy MediLogue onto native Alibaba Cloud serverless architecture for instantaneous global scalability.
  • Offline Edge Compute: Optimizing the Qwen core model pipelines using OpenVINO to allow MediLogue to run locally on consumer-grade high-performance hardware in deep rural clinics with zero internet connectivity.

Built With

Share this project:

Updates