MedInstructAI – Smart Healthcare Explainer
Inspiration In smart cities, access to healthcare is expanding — but medical reports are still written in highly technical language that most patients cannot understand. In rural and urban communities alike, this creates a communication gap between doctors and patients. I wanted to build something that could bridge this gap using Generative AI, by simplifying medical reports into clear, understandable language.
What it does Takes a medical report (PDF, text, or scanned image). Uses OCR + LLM (via Ollama) to read and understand the report. Generates a layman-friendly explanation of the findings. Translates the explanation into multiple languages and provides voice output so patients can hear it in their own dialect. Allows follow-up Q&A, so patients can ask clarifying questions about their report.
How we built it OCR: Extracted text from PDFs/images. LLM (Ollama locally): Used lightweight open-source LLMs Mistral-7B for medical comprehension and simplification. Multilingual Output: Integrated translation and text-to-speech for regional language voice support. UI: Built a Streamlit interface for user-friendly interaction.
Challenges we ran into Running large language models on limited hardware. Optimizing the pipeline so everything works offline, without relying on paid APIs. Handling long, complex reports within small LLM context windows. Recording a clean demo video within time constraints. Accomplishments that we're proud of Built a working end-to-end AI system that explains medical reports in simple terms. Achieved multilingual voice support for accessibility. Designed the project to work locally and offline, making it usable in rural or low-connectivity areas. Showcased how Generative AI can directly improve healthcare access in smart cities.
What we learned How to integrate OCR, LLMs, and TTS into one workflow. The importance of designing for resource-constrained environments. That impact often matters more than complexity — simple solutions can transform patient experiences.
What's next for MedInstructAI – Smart Healthcare Explainer Add better medical risk detection (flagging potential critical issues in reports). Improve UI/UX for doctors and patients. Deploy on the cloud for wider accessibility while keeping an offline fallback mode. Extend beyond reports to include prescriptions and lab results.
Log in or sign up for Devpost to join the conversation.