PillSnap – Instant AI Pill Identifier for HCPs

Inspiration

At Ng Teng Fong General Hospital, a senior pharmacist described medication reconciliation as “detective work.” A single patient on multiple drugs can take hours to investigate. With 95% of elderly Singaporeans experiencing polypharmacy and over half not literate in English, misidentification is a daily risk. We wanted to create a tool that transforms this detective work into seconds of clarity.

What it does

PillSnap lets healthcare professionals snap a photo of loose or unlabelled pills. Using machine learning segmentation and AI-powered identification, the system detects each pill and provides the top 3 confidence matches, with details on imprint, color, size, and typical use cases. It also explains its reasoning, helping clinicians make safe and confident decisions quickly.

How we built it

• YOLOv8 for pill detection and segmentation.
  
• OpenAI GPT-4o mini for pill identification, reasoning, and generating clear medication descriptions.
  
• Next.js frontend for the demo platform.
  
• Vercel for backend hosting.
  
• v0 for quick prototyping.

Challenges we ran into

• Getting accurate segmentation of cluttered pill images.
  
• Ensuring the AI provided detailed but clinically relevant reasoning (avoiding hallucinations).
  
• Time constraints in integrating multiple APIs into a smooth workflow.
  

Accomplishments we’re proud of

• A working demo where pills are segmented and identified within seconds.
  
• Strong validation from healthcare professionals.
  
• A clean, intuitive interface that mirrors real clinical workflows.
  

What we learned

• How to balance speed and accuracy in computer vision + LLM pipelines.
  
• The importance of user validation early in healthcare projects.
  
• That healthcare AI adoption depends on trust and every design choice must inspire confidence.
  

What’s next for PillSnap

• Integrating drug safety alerts and interaction checks.
  
• A professional forum with AI-powered case summaries.
  
• EHR integrations for seamless hospital use.
  
• Expanding beyond Singapore to Southeast Asia and global healthcare markets.

Built With

• next.js
• openai
• typescript
• v0
• vercel
• yolov8