AegisMedix

🩺 Inspiration

The inspiration for AegisMedix came from a simple question: Can we move AI from behind a chat window and put it "in the room" with a patient? Most digital health tools are passive—they wait for you to log data. I wanted to build a "Sentinel"—an autonomous companion that uses the eyes and ears of Gemini 3 to proactively participate in a patient's recovery.

🛡️ What it does

AegisMedix is an autonomous medical sentinel that manages the complexity of recovery so patients don't have to.

Live Health Bench: Conduct real-time voice/video consultations where Dr. Aegis identifies medications held to the camera and correlates them with patient history.
Proactive Recovery: An agentic engine that summarizes sessions, updates recovery protocols, and tracks adherence in real-time.
Sentinel Monitoring: A background system monitoring vitals and medication adherence, triggering SOS alerts if anomalies are detected.

🛠️ How I built it

Intelligence: Architected around Gemini 3 Live API (Multimodal Bidi). This allows for native, low-latency processing of audio and video frames in a single inference session.
Backend (Cortex): FastAPI server managing the Gemini proxy, background SMTP tasks, and secure state persistence.
Frontend (Desktop): A premium Next.js interface with medical-grade dark aesthetics and real-time WebSocket streaming.
Database: Supabase (Postgres) for secure medical history and session logs.

🚩 Challenges I ran into

Syncing high-frequency audio streams with real-time video frames while maintaining a coherent medical personality was the biggest hurdle. I had to optimize frontend audio buffering and implement a custom noise gate to ensure the AI's "hearing" was as crisp as its visual reasoning.

🏆 Accomplishments that I'm proud of

The "Live Video Consultation" is a breakthrough. Seeing Gemini 3 identify a medicine bottle via camera and immediately reason about its interaction with a patient’s unique vitals—with zero lag—was a true "wow" moment.

📖 What I learned

I learned that the future of healthcare isn't just about gathering better data; it's about better reasoning. Native multimodality changes everything for remote diagnostics and autonomous care.