MediBot

🏥 Inspiration

In high-stress medical emergencies, panic often prevents bystanders from acting effectively. We built MediBot to bridge the critical gap between an injury occurring and professional help arriving, providing both the physical tools and the expert guidance needed to save lives.

🤖 What it does

MediBot is a comprehensive emergency response ecosystem:

Smart Detection: Utilizing a Raspberry Pi and CSI camera, the system generates a voxel map to monitor for falls and uses Vision AI to instantly classify injuries like burns, lacerations, or sprains.
Robotic Assistance: Once an injury is identified, a custom robotic arm integrated with a first aid kit automatically retrieves and presents the exact medical supplies needed for that specific situation.
AI Voice Guidance: Powered by LiveKit and ElevenLabs, MediBot provides calming, step-by-step vocal instructions, ensuring the user stays focused while performing first aid.

🛠️ How we built it

The intelligence of MediBot relies on a high-performance tech stack:

Vision: OpenAI GPT-4 Vision and Google Gemini for real-time injury classification.
Voice: A low-latency pipeline using Deepgram for STT and ElevenLabs for natural-sounding TTS.
Orchestration: LiveKit Agents manage the real-time stream processing, while Groq (Llama 3.3 70B) handles lightning-fast decision-making and instructions.
Hardware: Python-based control for the robotic arm and Raspberry Pi for environmental mapping.

🚀 Challenges we ran into

Integrating a robotic hardware component with real-time AI vision required significant fine-tuning. We focused heavily on reducing latency between the vision classification and the voice agent to ensure instructions were delivered without life-threatening delays.