-
-
toothfairy1
-
toothfairy2
-
toothfairy3
-
toothfairy4
-
toothfairy5
-
toothfairy6
toothfairy
One click. Every diagnosis. Zero tab switching.
toothfairy is a cursor-inspired, AI-powered dentistry assistant built for both sides of the chair. For dentists, it brings X-ray analysis, clinical notes extraction, and treatment planning into a single intelligent workspace, streamlining + syncing workflows (i.e. removing the need for doctors to grab physical documents).
For patients, it replaces confusion and paper printouts with a visual, plain-language dashboard they can check from home. It remembers patient records across every visit, giving clinicians and clients the same complete picture: current findings, treatment history, and what comes next :) (currently in the build process for the patient-side!)
Inspiration 💡
Dentists deal with way too many disconnected tools every day: practice management software, separate imaging viewers, static paper charts, CDT code references, insurance portals. None of them talk to each other and none of them remember the patient properly (context!!!!!). There's no way to quickly see "tooth #36 had a cavity six months ago, we filled it, and now there's a periapical lesion forming."
But there's another problem nobody talks about: the patient has no idea what's going on. You go to the dentist, they point at a blurry X-ray and say "see that shadow?" and you just nod. Then you go home and couldn't tell someone what's wrong with your teeth if you tried.
For dentists: Upload an X-ray or highlight clinical notes, and toothfairy extracts findings, lights up affected teeth on an interactive chart, and builds a treatment timeline with CDT codes and costs. It pulls in history from past visits and handles the boring stuff so the dentist can focus on clinical judgment. Shorter appointments, more patients per day.
For patients: Color-coded teeth, plain descriptions, treatment timelines, cost estimates! Pull up your dental history at home and see how things have changed. 😌
What it does ✨
| Feature | Description |
|---|---|
| Clinical Notes Extraction | Gemini 2.5 Flash extracts structured findings from messy clinical notes: tooth numbers, conditions, severity, confidence scores. Maps to CDT codes and cost estimates. |
| X-Ray Analysis | Click a tooth in a panoramic X-ray view, toothfairy's complete ML pipeline (U-Net semantic segmentation model trained on panoramic dental X-rays) segments the boundary. Gemini Vision flags cavities, bone loss, periapical lesions. |
| Interactive Tooth Chart | SVG chart of all 32 teeth and a 3D model, both color-coded in real time. Yellow for cavities, red for lesions, purple for bone loss. |
| Treatment Planning | Evidence-based treatment info: success rates, risk factors, alternatives, referral notes. |
| Client Memory 🧠 | Moorcheh AI gives every patient their own memory namespace, semantically searchable across visits. |
| Patient Self-Service | Clients view their own chart, understand conditions in plain language, track changes across visits. |
Example Userflows!!!
You cmd + click tooth #36, #14, + #47.
Tooth Fairy AI: "Mderate periapical lesion on tooth #36,
early bone loss on tooth #14, deep cavity on tooth #47."
*Teeth #36, #14, and #47 light up on the chart and 3D model in their respective colors*
*Treatment timeline appears: Root canal for #36, scaling for #14, filling for #47*
A patient checks their dashboard at home after a visit.
They see their tooth chart with #36 highlighted red.
They click it: "Periapical lesion (moderate). Recommended: Root canal therapy.
Previous visit: cavity was detected 6 months ago on this same tooth. (ahhhh, now you see we're getting somewhere!"
How we built it 🛠️
┌─────────────────────────────────────────────────────────────────────────────┐
│ TOOTH FAIRY AI PIPELINE │
└─────────────────────────────────────────────────────────────────────────────┘
🦷 User Input (Panoramic X-Ray, Clinical Notes)
│
▼
┌─────────────┐ ┌─────────────┐ ┌──────────────────────┐
│ Next.js 16 │────▶│ FastAPI │────▶│ Moorcheh AI │
│ React 19 │ │ (Python) │ │ (Patient Memory) │
└─────────────┘ └─────────────┘ └──────────────────────┘
│
┌─────────┼──────────────┐
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────────────┐
│Imaging │ │ Notes │ │ Treat. + Meds │
│Copilot │ │Copilot │ │ Copilot │
└────────┘ └────────┘ └────────────────┘
│ │ │
│ │ │
┌─────────▼─────────┐ ┌─────▼──────────────┐
│ Tooth Segmentation│ │ PharmacyMCP │
│ (Custom CV Stack) │ │ (Health Canada │
│ │ │ DPD API) │
│ U-Net (TensorFlow)│ └────────────────────┘
│ ↓ │
│ Binary Mask │
│ ↓ │
│ OpenCV CCA │
│ ↓ │
│ Tooth Contours │
└─────────┬──────────┘
│
▼
┌──────────────┐
│ Gemini Vision│
│ Pathology AI │
│ Cavities │
│ Bone loss │
│ Lesions │
└──────────────┘
│
▼
┌──────────────┐
│ Protocol + │
│ Evidence Map │
│ CDT Codes │
└──────────────┘
│
▼
┌──────────────┐
│ SSE Stream │
│ (Live Logs) │
└──────────────┘
│
▼
🦷 Visual Output
(SVG Tooth Chart + 3D Model + X-Ray Overlays + Timeline)
Architecture Overview
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | Next.js 16, React 19, TypeScript, Tailwind CSS 4 | Dashboard UI with three-pane IDE layout |
| 3D Viewer | Three.js, React Three Fiber, Drei | Interactive 3D tooth model with finding highlights |
| Backend | FastAPI, Python, Pydantic | API routing, copilot orchestration, session management |
| LLM | Google Gemini 2.0 Flash | Structured JSON extraction from clinical notes |
| Vision | Google Gemini Vision | Pathology detection in X-ray regions |
| Segmentation | TensorFlow/Keras (U-Net), OpenCV (Connected Component Analysis) | Semantic segmentation of panoramic dental X-rays followed by isolation of individual teeth |
| Contour Extraction | OpenCV | Converts segmentation masks into tooth contour polygons rendered as SVG overlays |
| Medication Lookup | PharmacyMCP + Health Canada DPD API | Real-time drug product and DIN lookup based on detected findings |
| Patient Memory | Moorcheh AI SDK | Longitudinal history across sessions |
| Streaming | SSE (sse-starlette) | Real-time log streaming to frontend |
| Image Processing | Pillow, NumPy | X-ray manipulation and region cropping |
ML and Backend (FastAPI + Python)
PharmacyMCP: Health Canada Drug Product Database Integration When Tooth Fairy detects a dental finding, it queries the Health Canada Drug Product Database (DPD) via PharmacyMCP to surface relevant medications in real time. PharmacyMCP is an MCP server that wraps Health Canada's public DPD API, enabling structured drug lookups by condition. Once a finding is confirmed, the treatment copilot calls PharmacyMCP to fetch drug products with their official DIN numbers (unique identifier Health Canada assigns to every approved drug in Canada).
Custom Tooth Segmentation Pipeline:
Instead of relying on a generic medical segmentation API, we refactored a custom dental radiograph segmentation pipeline from an open source research project (TUFTS Research). Panoramic X-rays are first processed by a U-Net semantic segmentation model (TensorFlow/Keras) trained on dental radiograph datasets to generate a binary tooth mask. The mask is then processed using OpenCV connected component analysis (CCA) to isolate individual teeth. Finally, OpenCV contour extraction converts each tooth region into polygon boundaries that are rendered as SVG overlays in the X-ray viewer, enabling precise tooth-level interaction.
Gemini Vision Pathology Detection: After segmentation, we crop the region around the clicked tooth and send it to Gemini Vision with a dental radiologist system prompt. Returns structured findings with condition type (cavity, periapical lesion, bone loss, impaction), severity rating, and confidence scores, mapped directly onto the tooth chart.
Clinical Notes NLP Pipeline: Raw unstructured notes are sent to Gemini 2.5 Flash with a structured extraction prompt that outputs JSON arrays of [{tooth_number, condition, severity, confidence}]. A robust parsing layer strips markdown fences, handles random preamble text, and validates malformed responses. Regex-based extraction acts as a fallback when the model output deviates from the expected schema.
CDT Protocol Mapping Engine: protocol_mapper.py maps extracted dental findings to standardized treatment protocols. For example, "periapical_lesion" maps to Root Canal Therapy (CDT D3310) with estimated costs ($800–$1200) and expected visit counts. The engine covers common dental conditions and attaches evidence-based recommendations, referral guidance, and treatment sequencing.
Tooth Coordinate Mapping: tooth_mapper.py translates click coordinates on panoramic X-rays to FDI tooth numbers. Real panoramic X-rays have the upper arch at roughly 20-45% image height and lower arch at 55-80%. Thresholds were tuned against real X-ray dimensions and account for uneven spacing between molars, premolars, and incisors.
Moorcheh AI Client Memory: Every patient gets their own isolated namespace in Moorcheh AI. At session end, we automatically ingest findings, summaries, and treatment plans. On return visits, semantic search retrieves relevant history instantly. The same memory powers the patient-facing dashboard. Namespace-per-patient design means zero data leakage.
Dual-Summary Generation: Every finding gets generated in two versions: clinical (CDT codes, confidence scores, proper terminology) for the dentist, and plain language ("you have a cavity") for the patient.
Live-First Reliability Architecture: Custom ReliabilityManager wrapping every external call with configurable time budgets and automatic fallbacks. Gemini returns garbage? Regex extractor. Moorcheh not configured? Works without history.
SSE Terminal Streaming: Server-Sent Events via sse-starlette streaming copilot logs to the frontend in real time with explicit fallback notifications.
Copilot State Orchestration: Imaging, notes, and treatment copilots update patient state simultaneously. Session manager gives each copilot its own state slice so updates don't conflict, enabling cross-copilot handoffs.
Frontend (Next.js + Tailwind)
IDE-Style Three-Pane Layout: Light-mode workspace with Left (Patient Repository + mini tooth chart), Center (Active Workspace), and Right (Agent Terminal with live logs). Built with Next.js 16, React 19, TypeScript, Tailwind CSS 4.
Interactive SVG Tooth Chart: Custom React component rendering all 32 teeth as SVG paths using FDI numbering. Color-codes in real time: #F4C152 (yellow) cavities, #FF5C7A (red) lesions, #A78BFA (purple) bone loss, #4C9AFF (blue) impacted, #6E7A92 (gray) missing. Teeth pulse on status change.
3D Tooth Viewer with Finding Highlights: Three.js with React Three Fiber and Drei. When copilots detect issues, affected teeth get highlighted on the 3D model with the same color-coding as the SVG chart. Users can rotate, zoom, and click individual teeth to inspect findings spatially.
X-Ray Viewer with Segmentation Overlay: Panoramic X-ray display with click-to-analyze. U-Net semantic segmentation + OpenCV contour extraction rendered as SVG overlays
Context-Selection Triggers: Click a tooth in the X-ray (imaging copilot), highlight clinical notes text (notes copilot), click a finding card (treatment copilot). Each dispatches to its specialized copilot with terminal streaming.
Patient-Facing Dashboard: Same underlying data in plain language. Color-coded charts, readable descriptions, treatment timelines with costs, visit-over-visit progression. Powered by the same Moorcheh namespaces the dentist uses.
🚧 Challenges we ran into
Getting Gemini to output clean JSON. Sometimes nice JSON, sometimes markdown code blocks or random commentary. Added a parsing layer that strips fences and hunts for the JSON array, with regex extraction as fallback when the API is down.
Vision model latency. While the U-Net segmentation and OpenCV contour extraction run quickly (~1–2 seconds), pathology detection relies on Gemini Vision and can take several seconds depending on API response time. We built a ReliabilityManager that wraps external calls with configurable time budgets and fallback strategies. If the vision model times out or fails, the system still returns segmentation overlays immediately and surfaces the issue in the live terminal logs so the user never experiences a silent failure.
Tooth mapping from click coordinates. Real panoramic X-rays have arches at 20-45% and 55-80% image height (not 50/50). Left-right mapping within each arch was tricky too because teeth aren't evenly spaced. Lots of threshold tuning in tooth_mapper.py.
Two audiences. Dentists want CDT codes and confidence scores. Patients want "you have a cavity" not "carious lesion detected on the occlusal surface." We generate two versions of every summary automatically.
Per-patient data isolation. Each patient ID gets sanitized into a namespace, all documents live there, searches only query one namespace at a time. Getting deduplication and error handling right took a few iterations.
🏆 Accomplishments that we're proud of
✅ Three copilots end to end: notes to chart to treatment plan ✅ ML Pipeline for segmentation on teeth X-rays ✅ Gemini Vision for automated pathology detection with confidence scores ✅ Moorcheh AI with per-patient namespaces, fully isolated and semantically searchable ✅ Reliability system with cached/computed fallbacks for every external service ✅ Dual audience design: clinical precision for dentists AND plain language for patients
📚 What we learned
- Building around the client, not the feature. Making the patient the central organizing unit was our best decision. Making the patient an actual user (not just the subject) changed how we thought about everything.
- Always have a fallback. Every external service has a backup path. The app should never just crash. -MCP servers are a powerful integration primitive. PharmacyMCP showed us how wrapping a government API as an MCP server makes it trivially composable with AI pipelines — what would've been a custom API client became a plug-and-play tool.
- Empowering patients is a feature, not an afterthought. Most dental software treats the patient like a passive participant. Making them informed changed the whole product.
What's next for Tooth Fairy 🚀
| Feature | Status | Description |
|---|---|---|
| CBCT 3D Support | 🔜 Planned | Extend the 3D viewer to handle cone-beam CT scans |
| Voice Notes | 🔜 Planned | Dentists dictate during procedures, Tooth Fairy transcribes and extracts live |
| Insurance Pre-auth | 💡 Concept | Auto-generate pre-authorization letters from treatment plans and images |
| Family Accounts | 💡 Concept | Parents view and manage dental records for their kids in one place |
The Dream: A single interface where a dentist goes from raw X-ray to completed treatment plan to insurance submission, and where the patient goes home and actually understands what happened, what's coming next, and why it matters. 🎉
Log in or sign up for Devpost to join the conversation.