SmritiMeds is a hackathon-focused medication assistant that combines:
- React + Vite for a vibrant frontend
- FastAPI for the Python inference backend
- Anthropic Claude Vision for label understanding and reminder schedule generation
- medical OCR backends for printed and handwritten prescription extraction
- optional YOLO pill detection
- optional local Hugging Face pill classification
The name blends Smriti (Sanskrit for remembrance / memory) with Meds.
Current architecture
text frontend/ React + Vite UI backend/main.py FastAPI API anthropic_client.py Claude Vision + OCR-text normalization ocr_models.py Medical OCR routing pill_detector.py YOLO pill detection pill_identifier_model.py Hugging Face pill classifier integration local_pill_pipeline.py Hybrid local pill pipeline
Frontend
- vibrant gradient UI
- upload flow for:
- bottle / blister images
- printed medical documents
- handwritten prescriptions
- result panels for:
- OCR extraction
- structured medication schedule
- verification summary
- local vision beta status
Backend routing
1. Bottle / blister / pharmacy label
- primary backend: Claude Vision
- best for curved labels and packaging text
2. Printed medical documents / forms / lab reports
- primary OCR backend: naazimsnh02/medocr-vision
- then Claude converts OCR text into remin