medify.Rx

identify. medify. demystify

Comment

Inspiration

As students, we deal with a ton of health problems made worse by being away from home and feeling confused and alone. Doctors constantly use complicated medical terms that make a high heart rate sound like a heart attack. Not to mention, it can be incredibly difficult to figure out what medications should be combined and what shouldn't. We realized there had to be a better method of solution than constantly googling things.

What it does

Medify has three main features. “Medictionary” is used as a way to quickly answer questions about medications and other health-related issues. “Prescriptive” is the innovative tool to visualize the interactions between your medication, foods, and allergies to make sure you can stay safe. Finally, “Compremedic” is a tool to help you with understanding your medication and consent forms and to help you understand medical jargon right on the spot.

How we built it

Mobile (Expo / React Native): A live camera highlighter that runs ML Kit OCR on device, detects critical prescription fields, and overlays plain language explanations with text-to-speech. Photos never leave the phone. Web (Vite + React 19): Compremedic reads prescription labels from photos, PDFs, and Word documents entirely in the browser using Tesseract, pdf.js, and mammoth. Prescriptive renders a 3D drug interaction tree using three.js, WebXR, and MediaPipe hand controls. Medictionary provides medication Q&A. Server (Express + MongoDB): Handles authentication, accounts, validation, drug lookups through RxNorm and openFDA, and sourced interaction data. Google Gemini powers the AI features, including Medictionary chat, plain language rewrites, and interaction explanations. A placeholder system protects critical values like dosages and dates, with a glossary fallback if the output looks unsafe.

Accomplishments that we're proud of

We're proud of implementing real-time hand tracking, which lets users interact with Medify and their system of medications in a natural and novel way. We also built an interactive 3D graph that makes complex medical information easier to explore and understand visually. Most importantly, every insight Medify presents is grounded in evidence-backed research, so users can trust that the information comes from credible sources.

What we learned

This project was several of our team members' first deep dive into WebXR, and we learned how to build immersive experiences that run directly in the browser. Along the way, we figured out how to handle spatial interactions, render 3D content efficiently, and design interfaces that feel intuitive in a mixed-reality environment. We also learned a new web deployment strategy, including registering a Namecheap domain name and deploying the front and back end using Vercel and Render. Our team was also new to iOS app development.

Literature review

The premise of our project was based on the idea that patients frequently misunderstand common medical phrases. This idea was backed by the following article: Accuracy in Patient Understanding of Common Medical Phrases. Based on 215 respondents, several common phrases were misunderstood when used in a medical setting, with the interpreted meaning frequently the exact opposite of what was intended. Cyberchondria is an excessive and/or repeated online health search associated with increased distress or health anxiety and persists despite interference with functioning and negative consequences. It can even affect regular relationships with physicians. This is detailed further in Keeping Dr. Google under control: how to prevent and manage cyberchondria. Medify reduces this affliction by keeping responses very concise. Finally, the nocebo effect is the idea that negative expectations deriving from a clinical encounter can produce negative outcomes. This can be seen when patients focus on negative side effects of their medication, as mentioned in The nocebo effect and its relevance for clinical practice. Our application gives users the ability to control the level of detail they receive from their queries, preventing unnecessary strife.

What's next for Medify

Next, we plan to integrate a retrieval-augmented generation (RAG) model, so Medify can pull from its research database and generate clear, conversational explanations. This would let users ask questions in plain language and receive accurate answers backed by the same trusted sources that power the rest of the app. We would also like to add patient prompting reminders to take a break and cut any possible spiraling short.

Built With

Share this project:

Updates

Submission history