Health lens

• 

  scan complete

• 

• 

  Processing file

• 

  uploading file

• 

• 

  Health lens home page

Inspiration

Many patients struggle to understand their prescriptions because labels use tiny text and medical jargon, while leaflets are dense and filled with irrelevant details. Practitioner factors like illegible handwriting and vague instructions make the problem worse, creating confusion and risk. HealthLens solves this by acting as an instant translator, turning complex, hard-to-read medical information into clear, simple language that patients can trust.

What it does

HealthLens eliminates the guesswork by turning any medication text—from a scrawled note to a dense leaflet—into clear, simple, 5th-grade level instructions, instantly promoting safety and health literacy.

How we built it

HealthLens was built with a simple three-part stack to quickly turn medical images into clear text. The frontend uses HTML, CSS, and JavaScript to capture images and convert them into Base64 strings. The backend (Flask + Pillow) processes the image, and the AI core (Gemini ) reads and simplifies the medical content, returning plain text to the user for instant display.

Challenges we ran into

We faced and fixed three major issues during development. First, an API key detection error was solved by properly setting the environment variable in the same terminal session. Then, missing libraries were installed to fix the Flask module error, and finally, a CORS blockade was resolved by adding the flask-cors extension to the backend.

Accomplishments that we're proud of

Solved a Critical Health Literacy Gap: We built a working prototype that uses multimodal AI (Gemini) to transform complex, dangerous medical jargon into safe, clear, 5th-grade level instructions instantly.Achieved Universal Clarity: We successfully expanded the scope to handle any drug document—prescriptions, labels, boxes, and leaflets—proving the solution is comprehensive and robust.Delivered a Resilient MVP: We built a stable, end-to-end tech stack (Frontend $\leftrightarrow$ Flask $\leftrightarrow$ Gemini) from scratch and successfully debugged key integration issues (API keys, CORS) to deliver a working demonstration.

What we learned

From building HealthLens, I discovered how multimodal AI can turn images into clear, life-saving language to address civic challenges. The biggest obstacle wasn’t text simplification but tackling integration issues like CORS and environment variables. I also learned that a focused user experience and a strong AI prompt are key to building an effective MVP quickly.

What's next for Health lens

The next evolution of HealthLens moves beyond simplification to become a proactive, personalized AI health assistant that tackles the complex challenge of medication adherence and follow-up care.

Our focus is on building a dedicated AI Agent capable of:

Continuous Engagement: Following up with clients via their preferred channel—be it Email, WhatsApp, or other social media platforms—to check on their medication status and adherence.

Voice-Enabled Support: Integrating Speech Recognition to allow users to verbally confirm they have taken their medication or ask immediate questions, making the assistant fully accessible.

Automated Scheduling: Taking responsibility for care coordination by proactively booking follow-up check-ups and appointments with practitioners, and saving these events directly to the client's digital calendar.

This transition establishes HealthLens as a comprehensive, always-on partner in patient care, ensuring compliance and improving long-term health outcomes.

Built With

• css
• flask
• flask-cors
• html
• javascript
• python