MedSight AI

Inspiration

Patients often receive dense medical reports or diagnostic scans filled with complex clinical jargon. This creates high anxiety, confusion, and accessibility barriers—especially for patients with limited health literacy or vision impairments. We created MedSight AI to bridge this gap, turning complex clinical data into plain-language explanations and voice narration.

What it does

MedSight AI is a multimodal clinical assistant that:

Analyzes Diagnostic Scans & PDFs: Uploads chest X-rays, medical images, or PDF reports using Gemini 1.5 Flash.
Simplifies Jargon: Converts technical findings into clear, easy-to-understand summaries.
Highlights Key Metrics & Risks: Flags potential areas of concern and abnormal findings clearly.
Generates Audio Narration: Uses text-to-speech (gTTS) for regional audio playback to improve accessibility.
Empowers Patient-Doctor Conversations: Generates actionable questions for patients to ask during their next medical visit.

How we built it

Frontend UI: Built with Streamlit for a fast, responsive browser experience.
Multimodal Intelligence: Powered by Google's Gemini 1.5 Flash API for vision and text reasoning.
Image & Document Processing: Handled using Pillow (PIL) and pypdf.
Audio Generation: Integrated gTTS to render instant localized audio.

Challenges we faced

Fine-tuning prompts to ensure clinical accuracy while remaining clear and approachable for non-medical users.
Handling diverse image formats and document layouts seamlessly without latency.

What we learned

How to leverage Gemini 1.5 Flash's fast multimodal reasoning for real-time visual analysis.
Designing human-centered healthcare interfaces focused on accessibility and safety disclaimers.

What's next for MedSight AI

Adding support for multi-language translation across diverse regional languages.
Expanding diagnostic capabilities to handle historical report comparisons over time.