Medicare Vision AI

Inspiration

My grandmother often struggled with her tablet routine. The print on medication slips was too small, and she sometimes skipped doses or mixed up tablets. Watching this happen made it clear that many elderly people need a simple, dependable way to verify their medication without relying on others. That pushed us to build a system that sees the tablet through a camera, understands what it is, and compares it with the prescription automatically.

What it does

The system uses AI vision to identify a tablet from a live camera feed, match it against the user's prescription list, and confirm whether it is the correct medication for that time. If the user picks up the wrong tablet, the system triggers a clear warning. It also includes a built-in voice assistant to guide the user, making the experience hands-free and accessible.

How we built it

We developed the platform using React, JavaScript, and TypeScript for the front end. The AI vision pipeline handles:

• Image capture
• Tablet feature extraction
• Matching using an embedded model
• Verification logic against the prescription schedule

The interface is simple enough for elderly users but powered by a robust matching backend.

Mathematically, the matching problem can be expressed as:

$$ \hat{y} = \arg\min_{y \in \mathcal{P}} ; d(f(x), f(y)) $$

where

• ( x ) is the captured tablet image,
• ( \mathcal{P} ) is the set of prescribed tablets,
• ( f(\cdot) ) is the feature extractor,
• ( d(\cdot,\cdot) ) is a distance metric between embeddings.

Challenges we ran into

The toughest part was achieving reliable tablet recognition. Different lighting conditions, slight variations in angle, and similarity between certain tablets made the model unstable in early tests. Verifying the scanned tablet against the prescription in real time also required tuning both the model and the comparison logic.

Accomplishments that we're proud of

We built something that genuinely helps elderly people stay safe with their medication. A simple camera acts like a real human helper, guiding and correcting them when family members are busy or away. That practical impact is what we value most.

What we learned

This project taught us how to manage a large-scale build from start to finish. We gained solid experience in integrating AI vision with a production front end, handling real-time processing, and designing for accessibility.

What's next for Medicare Vision AI

We plan to scale the system, improve accuracy across more tablet types, and fully host the application so it can be used by anyone. Adding caregiver dashboards and prescription management features is also on the roadmap.

Built With

• bcryptjs-file-processing:-multer
• express.js-5-database:-sqlite-(better-sqlite3)-ai/ml:-google-gemini-ai-ocr:-tesseract.js-email:-resend-api
• javascript-frontend:-react-19
• languages:-typescript
• nodemailer-(smtp)-auth:-jwt
• sharp-development:-typescript
• vite
• vite-6-backend:-node.js