Inspiration

Medication non-adherence is a global healthcare challenge, especially among elderly patients and people managing multiple medications. Many patients forget doses, misunderstand medication instructions, or are unaware of potential drug interactions. Caregivers often struggle to monitor adherence and ensure medications are taken safely. We wanted to build a solution that transforms medication management from a confusing and stressful experience into a simple, guided, and safer process.

What it does

DoseMate is an AI-powered medication companion that helps patients understand, organize, and safely manage their medications.

Users can upload images of medication packages or prescriptions. Our AI identifies medications, retrieves trusted information, explains usage and side effects in simple language, answers medication-related questions, generates personalized medication schedules, detects potential drug interactions, sends reminders, tracks adherence, and keeps caregivers informed when doses are missed.

The goal is to improve medication safety, adherence, and patient confidence.

How we built it

DoseMate combines multiple AI components into a single workflow:

1. A Vision-Language Model (VLM) extracts medication names from images of medication boxes or prescriptions.
2. A Retrieval-Augmented Generation (RAG) system retrieves trusted medication information from our medication knowledge base.
3. Gemini 2.5 Flash Lite generates grounded, patient-friendly explanations and answers medication-related questions.
4. A scheduling engine creates personalized medication plans based on medication instructions and timing requirements.
5. A drug interaction module checks for known interactions and surfaces appropriate warnings.
6. A reminder and adherence tracking system monitors medication intake and can notify caregivers when doses are missed.

This architecture ensures that responses are grounded in verified medication information rather than relying solely on a language model.

Challenges we ran into

One of our biggest challenges was balancing AI flexibility with medical safety. We needed to ensure that medication information remained reliable while still providing natural and helpful responses.

Another challenge was designing a workflow that combines computer vision, retrieval systems, medication scheduling, and adherence tracking into a coherent user experience.

We also had to carefully handle potential drug interaction warnings and prevent the AI from generating unsupported medical advice.

Accomplishments that we're proud of

• Building an end-to-end AI medication management workflow.
• Successfully combining computer vision and RAG in a practical healthcare application.
• Creating a system that goes beyond answering questions by generating schedules, tracking adherence, and supporting caregivers.
• Designing the project with responsible AI principles and medical safety considerations in mind.
• Delivering a solution with real-world impact potential for elderly and chronic disease patients.

What we learned

Through this project, we learned that effective healthcare AI is not just about using large language models. Reliable systems require retrieval, verification, safety constraints, and thoughtful user experience design.

We also gained hands-on experience integrating computer vision, retrieval systems, reasoning models, workflow automation, and responsible AI practices into a single product.

What's next for DoseMate

Our next steps include:

• Expanding the medication knowledge base to support a much larger number of medications.
• Supporting multilingual medication assistance.
• Integrating with pharmacy and healthcare APIs.
• Adding voice-based interactions for elderly users.
• Enhancing caregiver dashboards and monitoring tools.
• Improving personalization based on patient medication habits.

Our long-term vision is to make DoseMate a trusted AI companion that helps patients safely navigate medication management and improve health outcomes.

Built With

• chroma
• ddi-algortihem
• fastapi
• huggingface
• langchain
• postgresql
• python
• rag
• react
• visionlangugaemodel
• vlm