MedInstructAI: AI Doctor That Teaches Health Literacy

• 

• 

• 

Inspiration

While volunteering in rural health camps, I noticed that most patients and even biology students couldn’t understand their own medical reports. Terms like hemoglobin, bilirubin, or ECG abnormalities sounded alien. This language gap not only affects patient awareness but also prevents students in rural or non-English-medium schools from learning medical science effectively. That’s where the idea for MedInstructAI was born — to make medical learning accessible, multilingual, and interactive through the power of AI. If AI can help doctors diagnose faster, why not help students learn medicine smarter?

What it does

MedInstructAI acts as a multilingual AI health educator that reads medical reports or text, understands them, and explains everything in simple, student-friendly language — both in text and voice. Key features include:

🧠 AI Understanding: Reads and explains medical concepts or reports.

🌍 Multilingual Output: Explains in Indian languages like Hindi, Bengali, Tamil, and Bhojpuri.

🗣️ Voice Learning Mode: Speaks the explanation aloud for students who prefer audio-based learning.

💬 Interactive Q&A: Students can ask follow-up questions to clarify doubts in real-time.

💻 Offline Support: Works with lightweight local models, even in low-connectivity regions.

In short, it turns complex health data into a personalized, language-inclusive classroom experience.

How we built it

Model Backend: Used Mistral 7B (GGUF) fine-tuned on open medical datasets for comprehension and explanation generation.

Local Serving: Deployed using Ollama + llama.cpp, ensuring fast and offline-ready inference.

Frontend: Built with Streamlit, offering a simple educational UI with text input, upload, and voice playback options.

Translation Layer: Integrated Google Translate API and open-source NLLB models for regional language support.

Speech Components: Added gTTS + Streamlit-webrtc for text-to-speech and future speech input capabilities.

OCR Integration: Used EasyOCR to read printed or handwritten medical reports for real-world use.

Every component was selected with one goal: bring AI-powered health education to resource-constrained learners.

Challenges we ran into

🧩 Limited Hardware: Running and fine-tuning large models locally on 8GB RAM was difficult — required optimizing models into GGUF quantized formats.

🌐 Translation Consistency: Maintaining accuracy across multiple Indian languages was challenging; some medical terms lacked direct translations.

🗣️ Voice Playback Sync: Getting the text-to-speech and Streamlit interface to work seamlessly required multiple iterations.

🧠 Simplification vs. Accuracy: Ensuring the AI simplifies without misrepresenting medical facts took careful dataset curation and prompt design.

Accomplishments that we're proud of

Successfully built a fully functional offline AI educator capable of explaining complex medical content in simple, spoken, multilingual form.

Enabled interactive question answering where students can continue the learning conversation.

Designed a user-friendly, minimal UI that can run even on low-end devices.

Created a prototype with real social impact — empowering medical literacy and education accessibility.

What we learned

How to fine-tune and quantize open-source LLMs like Mistral for specialized domains.

The importance of simplification, localization, and inclusivity in AI for education.

Integration of AI + Speech + Translation + UI into one smooth system.

How impactful it is when AI focuses on education, not automation — empowering people rather than replacing them.

What's next for MedInstructAI: AI Doctor That Teaches Health Literacy

🎙️ Add Real-Time Voice Input: So students can talk naturally with the AI instead of typing.

🌐 Build an Offline Mobile App Version: For Android devices used in rural classrooms.

🧑‍⚕️ Expand Curriculum: Include biology and healthcare topics for school and college learners.

🧩 Integrate Visuals: Add diagrams and AR-based medical illustrations to support visual learning.

🤝 Collaborations: Partner with NGOs and educational boards to deploy MedInstructAI in rural learning centers.

Ultimately, MedInstructAI aims to become the “AI Teacher for Health Literacy” — making medical education inclusive, engaging, and understandable for every student, in every language.

Built With

• easyocr
• gtts
• mistral7b
• ollama
• python
• streamlit
• translator