MediLogue

Inspiration

The communication barriers faced by rural patients and multi-dialect communities (such as local Sabah dialects) on the healthcare frontlines often delay emergency diagnosis. We were inspired to bridge this gap using artificial intelligence (AI).

What it does

Medilogue is an intelligent multimodal clinical triage console that automatically and rapidly translates dialect voice inputs, hand gestures, and patient biometric vitals into clinical summaries and treatment urgency levels.

How we built it

• Development & Environment: Built using React, TypeScript, Vite, and Tailwind CSS within a Cloud Sandbox environment accelerated by Codex.
• AI Integration: Utilized TensorFlow.js and handpose models for hand-gesture recognition, alongside intelligent logic architecture powered by GPT-5.6 and Codex.

Challenges we ran into

• Resolving external server real-time connection constraints and managing complex npm package dependency compatibility during initial setup.
• Ensuring the multimodal data flow between live video streams and phonetic dialect textfields functioned smoothly without TypeScript type errors.

Accomplishments that we're proud of

• Successfully creating a clean, responsive, and fully functional medical console interface in a short timeframe.
• Successfully integrating hand-gesture detection and local dialect phonetic text into a unified triage dashboard.

What we learned

• The importance of type-safe frontend architecture planning when combining multiple browser-based machine learning models.
• How workflow automation using AI assistants significantly accelerates bug resolution efficiently.

What's next for MediLogue

• Expanding the local dialect dictionary across various other regions in Malaysia.
• Integrating direct hospital API connections for safer and larger-scale patient record storage.

Built With

• codex
• fastapi
• gpt-5
• html5
• openaiapi
• python
• react
• tailwind
• typescript