MediPulse: Clinical Forensics Engine

Inspiration

Every year, millions of patients suffer from preventable Adverse Drug Events (ADEs). The core issue isn't a lack of medical knowledge; it's a data siloing problem. A patient visits a specialist who writes a new handwritten prescription. That specialist doesn't have access to the patient's recent blood test PDF from a different clinic, nor do they know what the patient plans to eat for dinner. Silent contraindications kill.

We were inspired by the massive multimodal context capabilities of Google Gemini 2.5 Flash to solve this. What if we could build a unified reasoning engine that digests completely unstructured, disparate documents—all at once—and acts as a live clinical pharmacologist?

What it does

MediPulse is a Multimodal Clinical Forensics Engine. It completely bypasses traditional forms and data entry. Patients or doctors simply upload what they have:

1. A photo of a handwritten prescription (JPG)
2. A dense, tabular lab report (PDF)
3. A photo of a meal they are about to eat (Image)

MediPulse feeds these simultaneously into the Gemini API, executing a deterministic cross-document contraindication check. It flags critical dangers (e.g., a prescription for Metformin contradicting a renal panel PDF showing an eGFR of 35). It features:

• Cross-Document Contraindication Engine: Catches conflicts between disparate medical documents.
• Multimodal Meal Safety Checker: Snap a photo of food to cross-reference against active medications.
• Live Multilingual Translation & Narration: Translates complex clinical jargon into regional languages (like Hindi or Tamil) and narrates it using the Web Speech API.

How we built it

• AI Core: We heavily utilized the new @google/genai SDK and the Gemini 2.5 Flash model. We leveraged its massive context window and native multimodal capabilities to parse PDFs, read handwritten JPGs, and analyze food images in a single payload.
• Deterministic Output: We utilized Gemini's responseSchema to force strict, heavily nested JSON structures so our frontend could reliably map the medical audit into interactive components.
• Frontend Architecture: Built a blazing-fast SPA using React 19 and Vite.
• Aesthetic Identity: We abandoned the generic "SaaS" look in favor of a premium, soft glassmorphism aesthetic with floating interfaces, vibrant gradients, and dynamic micro-interactions to create a calming yet highly advanced clinical forensics UI. We utilized Tailwind CSS for the sleek design and Framer Motion for snappy, hardware-accelerated scroll reveals and data-flow animations.

Challenges we ran into

• Multimodal Hallucinations: Initially, unstructured prompts led to inconsistent clinical advice. We spent hours refining the system prompt and enforcing strict JSON schemas to ensure Gemini acted deterministically and prioritized clinical safety constraints over conversational padding.
• Parsing Tabular Data: Ensuring the model correctly correlated a row in a PDF lab report (e.g., Creatinine levels) with a handwritten drug name in a separate image was a massive hurdle, solved entirely by Gemini 2.5's robust spatial reasoning.
• UI/UX: Striking the balance between an advanced 'clinical forensics' tool and an accessible, premium patient-facing application required careful typographic hierarchy, polished glassmorphism panels, and smooth aesthetic micro-interactions.

Accomplishments that we're proud of

• Successfully orchestrating a multi-document context window where a PDF directly influences the AI's understanding of a separate image.
• Developing the Meal Safety Checker—the UX of simply snapping a photo of Grapefruit and having the UI scream a warning about CYP3A4 Statin interactions feels like magic.
• Crafting a highly original, premium soft glassmorphism interface that stands out from typical AI template apps with its approachable, high-end feel.

What we learned

We learned the immense power of Google Gemini 2.5 Flash. The ability to throw raw PDFs, messy images, and text into a single API call and receive back cleanly typed, heavily structured JSON data fundamentally changes how medical software can be built. We no longer need OCR pipelines chained to NLP entity extractors; Gemini handles the entire reasoning stack.

What's next for MediPulse

• EHR Integration: Exploring FHIR standards to automatically pull patient history directly from hospital databases.
• Specialized Modules: Building specialized prompt architectures for Oncology and Pediatrics where dosing and interactions are even more critical.
• Mobile Native: Wrapping the application in React Native to allow seamless camera access for elderly patients.

Built With

• css3
• framer-motion
• gemini-api
• google-gemini-2.5-flash
• google-genai-sdk
• html5
• javascript
• ogl-webgl
• react
• tailwindcss
• vite
• web-speech-api