MedX

AI-powered 3D anatomy. Your body, fully explained

Comment

Inspiration

Every time someone described a symptom "my lower back hurts" or "sharp pain behind my eye" searching online led to panic, not answers. I wanted to build something that connects what you feel to what you see: a tool that takes plain English symptoms, maps them to the exact body part in 3D, and explains them with real medical data. No fluff. Just clarity.

What it does

MedX is a browser-based medical assistant. You type a symptom in natural language, and it:

  • Identifies the affected body part and highlights it on a full 3D human anatomy model
  • Matches your symptom to possible conditions using a medical knowledge base
  • Answers factual questions (e.g. "what is hypertension?") with definitions, causes, and treatments
  • Runs entirely client-side no server, no data leaves your machine

How we built it

  • Custom AI model Built and fine-tuned from scratch specifically for MedX, trained on medical Q&A datasets. Quantized to q4 and runs in-browser via Transformers.js. View the model on Hugging Face.
  • 3D Viewer Full human anatomy rendered in Three.js with dynamic region highlighting tied to AI output. Hundreds of OBJ files mapped to body-part names.
  • Knowledge Base Three layers: Symptom2Disease (symptom → disease), MedQuAD (disease → definitions/causes/treatments), and an anatomy encyclopedia searched via cosine similarity on MiniLM embeddings.
  • Frontend Vanilla HTML/CSS/JS with Bootstrap 5.

Challenges we ran into

  • *Browser AI * Running a language model in-browser without freezing the UI required q4 quantization and batched embedding processing with async yields between batches.
  • *Intent Misclassification * Medical questions like "why are my eyes twitching" kept hitting the wrong conversational intent handler. Fixed by using regex pattern matching for conversation intents instead of embeddings.
  • *Data Alignment * MedQuAD calls it "High Blood Pressure" but Symptom2Disease says "Hypertension". A reverse alias lookup bridged the gap so the educational card always shows.
  • *3D Alignment * Anatomy OBJ files came with a different coordinate system than Three.js expects. Days of transformation matrix tweaking to get smooth highlights.

Accomplishments that we're proud of

  • A fully functional AI-powered medical assistant that runs entirely in a web browser no backend, no API calls, no login required.
  • A custom fine-tuned language model deployed to Hugging Face and actively used by the app.
  • Seamless integration between AI output and interactive 3D anatomy the model highlights the right body part for every symptom.
  • Successfully processing and cross-linking two large medical datasets (MedQuAD + Symptom2Disease) into a unified knowledge base.

What we learned

  • How to quantize and deploy transformer models for client-side browser inference
  • How to build real-time 3D web applications with Three.js
  • How to design a hybrid search pipeline combining embeddings, keyword matching, and alias resolution
  • That intent classification with embeddings is fragile sometimes simple regex patterns outperform deep learning

What's next for MedX

  • Support for multiple languages
  • Voice input for symptom descriptions
  • A richer 3D experience with layered (muscular, skeletal, nervous system toggles)
  • Community-contributed symptom data to expand coverage
  • Mobile app version with offline model support

Built With

Share this project:

Updates

Submission history