MediScan

Inspiration

Every day, millions of people face a common dilemma: "Should I go to the hospital or rest at home?" We built MediScan to solve this — a real-time AI medical triage assistant that helps people make that decision instantly, using voice, text, and camera.

What it does

MediScan is a real-time AI medical triage assistant powered by Google Gemini. Users can:

Describe symptoms via text or voice in natural conversation
Show affected areas via live camera for visual AI analysis
Receive instant triage guidance: GREEN (home care), YELLOW (see doctor), RED (emergency)
Download a PDF medical report summarizing the entire session

How we built it

We built MediScan using:

Google Gemini 3 Flash for real-time text and symptom analysis
Gemini 2.5 Flash Native Audio Live API for real-time voice conversation
Gemini Vision for camera-based visual symptom detection
Python + Gradio for the web interface
FPDF2 for PDF report generation
OpenCV + PIL for camera capture and image processing

Challenges we ran into

Gemini Live API is not available in India — we used VPN to connect to US servers
API quota exhaustion during development forced us to rotate multiple API keys
Audio interruption issues when mic picked up agent's own voice — solved with speaking event locks
Gradio version compatibility issues with Python 3.14

Accomplishments that we're proud of

Built a fully working real-time voice + vision + text medical triage agent in under 2 days
Successfully integrated Gemini Live Audio API with real-time microphone streaming
Camera visual analysis correctly identifies visible symptoms and provides triage
PDF report generation works end-to-end

What we learned

How to use Gemini Live API for bidirectional audio streaming
How to handle async audio streams in Python
How to integrate multimodal AI (text + voice + vision) in a single application

What's next for MediScan

Add multilingual support for regional Indian languages
Integrate with Google Maps to find nearest hospitals for RED triage cases
Add patient history tracking across sessions
Deploy on cloud for public access

Built With

fpdf2
gemini-live-audio-api
gemini-vision
google-gemini-api
gradio
python

Submitted to

Gemini Live Agent Challenge

Created by

As the Lead Developer and Backend Engineer, I was responsible for the core architecture and AI integration of MediScan.

My contributions:
- Designed and built the complete project architecture from scratch
- Integrated Google Gemini 3 Flash API for real-time symptom analysis and medical triage
- Implemented Gemini 2.5 Flash Native Audio Live API for real-time bidirectional voice conversation
- Built the async audio streaming pipeline using Python sounddevice for microphone input and speaker output
- Developed the main agent loop handling text, voice, and camera inputs simultaneously
- Integrated Gemini Vision API for camera-based visual symptom detection
- Set up the project structure, environment configuration, and API key management
- Led the team of 3 members, coordinated task division between backend, camera/logic, and UI
- Debugged and resolved complex issues including API quota limits, audio interruption, and async stream management
- Merged all team members code into final working application

Abhijeet Kulkarni
Shreya Borkar
stephen kurane