MediSimplifier: Intelligent Medical Report Decoder

Democratizing health literacy by transforming complex lab reports into actionable, visual, and interactive insights using ERNIE 4.0 and PaddleOCR.

Comment

Inspiration

The "Golden Hour" in healthcare is critical, yet millions of patients struggle to understand their own medical data. A standard lab report is often just a wall of numbers—confusing, anxiety-inducing, and unintelligible to the layperson.

I asked myself: "What if your lab report could talk to you?"

That question inspired MediSimplifier. I wanted to build a "Medical Digital Twin" that doesn't just digitize data but democratizes it—turning complex medical jargon into simple, actionable, and visual insights that anyone, regardless of language or literacy level, can understand.

What it does

MediSimplifier is an intelligent, multimodal application that bridges the gap between raw medical data and patient understanding.

  1. Instant Digitization: Users upload a photo or PDF of a lab report. We use PaddleOCR to extract text with high precision.
  2. Intelligent Analysis: The data is fed into ERNIE 4.0, which acts as a "Medical Translator," identifying abnormal values, explaining them in simple English or Hindi, and generating a personalized diet/action plan.
  3. Visual Storytelling: Instead of tables, users see "Health Cards" with dynamic Gauge Charts (Red/Green zones) that make health status instantly understandable.
  4. Dr. ERNIE Chatbot: A context-aware agent that allows users to ask follow-up questions like "Can I eat sugar with this glucose level?", powered by the ERNIE Bot API.

How I built it

I architected a robust pipeline focusing on Accuracy and User Experience:

  • The Vision Layer (PaddleOCR): I utilized PaddleOCR-VL for its superior ability to handle complex table layouts in medical PDFs, converting raw pixels into structured text.
  • The Intelligence Layer (ERNIE 4.0): I engineered strict "Safety Guardrails" in our system prompts. We force ERNIE to output structured JSON data, ensuring the app never "hallucinates" a medical diagnosis but strictly interprets the provided numbers.
  • The Verification Layer: Recognizing that OCR isn't 100% perfect, we built a Human-in-the-Loop Verification Table that allows users to correct data before it goes to the AI.
  • The Experience Layer: Built with React + Vite and styled with Tailwind CSS, featuring a custom "Glassmorphism" design system to instill a sense of trust and modernity.

Challenges i ran into

  • The "Hallucination" Risk: Early tests showed LLMs sometimes inventing values. I solved this by implementing a Zod-based Schema Validation in our backend. If ERNIE's output doesn't match the strict JSON schema, the system automatically retries.
  • Complex Table Parsing: Medical reports vary wildly in format. Mapping raw OCR text to structured "Test/Value/Unit" objects was difficult. I used a hybrid approach of regex post-processing and ERNIE's reasoning capabilities to normalize this data.
  • Deployment Constraints: Hosting a heavy OCR model requires GPU. For the hackathon demo, we created a "Simulation Mode" that runs on the client-side for stability, while our video demonstration showcases the full local Python backend in action.

Accomplishments that i am proud of

  • The "Hallucination Firewall": I successfully created a prompting architecture that refuses to give medical advice if data is missing, ensuring the app remains a responsible information tool rather than a risky diagnostic one.
  • Visualizing Health: Moving away from Excel-style tables to Interactive Gauge Charts was a huge UX win. It turns abstract numbers into immediate understanding.
  • Seamless Multimodal Flow: Successfully chaining PaddleOCR (Vision) $\rightarrow$ Python Middleware $\rightarrow$ ERNIE 4.0 (Language) into a unified experience felt like magic when it first worked.

What i learned

  • Constraints are Key: I learned that in Healthcare AI, constraints are as important as capabilities. Writing prompts that refuse to diagnose was a key learning curve.
  • Trust is Visual: A medical app cannot look "hacky." We learned that investing time in a high-quality Glassmorphism UI was essential to making the AI feel authoritative and trustworthy.

What's next for MediSimplifier: Intelligent Medical Report Decoder

  • Voice-First Interaction: Adding voice support so elderly patients can simply speak to their reports in their local dialect.
  • Wearable Integration: Connecting with Apple Health/Fitbit to correlate lab results with daily activity data (e.g., "Your cholesterol is high, but your step count is improving").
  • Doctor Dashboard: A provider-facing view that summarizes these patient-friendly insights for doctors to review before appointments.

Built With

Share this project:

Updates