Snap & Safe

Snap&Safe: Identify pills, get medication info, set calendar reminders, and decode nutritional facts from photos.

Comment

Inspiration

The inspiration behind Snap&Safe came from seeing our family members struggle with reading nutritional and medical labels due to old age or simply the small and cluttered font when they needed to know certain nutritional facts or medicinal informations due to allergies or other dietary restrictions.

What it does

Snap&Safe allows users to take photos of pills, pill bottles, or boxes, and uses Gemini AI API to identify the medication. It also features a chat box for additional medication information, calendar reminders for medication schedules implemented with a Fetch.AI agent along with Google Calendar API, and nutritional facts extraction from bottle photos.

How we built it

We built Snap&Safe using React for the front end, JavaScript for client-side logic, Python with Flask for the backend, and Gemini AI API for medication and nutrition label identification. We also used a Fetch.ai agent to add events to google calendar along with Google Calendar API.

Challenges we ran into

Some challenges we encountered included integrating the Gemini AI API for accurate medication identification since it was new for all of our team members and implementing a user-friendly interface that would be accessible. We also struggled with implementing Flask as we had never used it before.

Accomplishments that we're proud of

We are proud of learning to use Figma to design the UI/UX. We are also proud of how much we have improved in React. Some of us have previous experience with Hackathons and feel like we were able to implement the Figma designs a lot faster and more accurate than before. For one of us, this was their first hackathon, and they are extremely proud throwing themselves into this new challenge even though they were nervous. We are especially proud of the teamwork our team had. We were able to collaborate with each other and coordinate tasks very efficiently. We were also very vocal with our ideas and this led us to improve on iterations for our project!

What we learned

Throughout the development process, we learned valuable skills in integrating APIs, managing state in React, creating a responsive web application, and collaborating effectively as a team. Additionally, we gained experience in using new technologies such as Gemini AI API , Fetch.AI agents and Flask.

What's next for Snap & Safe

In the future, we plan to enhance Snap&Safe by adding features such as medication interaction checking, medication expiration tracking, and user profile customization. We also aim to improve the accuracy and speed of medication identification using Gemini AI API. Additionally, we will continue to iterate on the user interface to make it even more intuitive and user-friendly.

Share this project:

Updates