PillPal

Inspiration

All four of us (the developers of this project) grew up with immediate family members who are immigrants and/or elderly. They are often prescribed all types of medicine to maintain their health. We all realized the similar struggles they share: trouble reading English and small text, differentiating between their medicines, and keeping track of when to take these medications.

Before embarking on this journey, we did research to see if someone had made a similar assistant in the past. To our surprise, the most we could find were Pharmaceutical companies seeking to make a profit off of disability (like CVS’s Spoken Rx, which requires you to use a “Spoken Rx prescription”). With the advent of AI and its increased accessibility, we see no reason why we can’t help those who are disadvantaged within our communities by utilizing these powerful tools.

We dedicate this project, PillPal, to our families.

What is PillPal?

PillPal is an app dedicated to users who have trouble understanding their medications and need to keep track of their prescriptions. Sometimes, we have very real concerns regarding whether or not our grandparents—or even we—can take certain medications given their medical history. Will this medication conflict with other medicines? Are there things that we should be aware of when taking these medications?

The traditional microscopic drug facts label and packets of legal documentation fail to provide an adequate understanding of drug use safety and fail to be transparent. We seek to utilize AI to provide increased understanding regarding what you’re taking, and when you should take it.

The user can access the camera to take a picture of their prescription bottle. PillPal will take the data from that picture and generate information for the user, specifically: the purpose of the medicine, considerations with taking it, the assigned dosage, and personalized advice based on user information. PillPal also has translation capabilities, with written advice written in the language of their choice, while also reading this consultation out loud. Users are also able to create an account, enter their medical information, and save their medications to their profile to easily keep track of their prescriptions which they can get personalized results.

Building & Learning Process

We used a MERN (MongoDB Atlas, Express.js, React Native, Node.js) stack for app development. We are all very new to both React Native and app development but since we have more web development experience, we tried our best to translate that into an app format since this project is more practical in mobile form. We used CI/CD as much as we were able to, especially when integrating the API’s. We integrated Google Cloud Vision API, Google Cloud Text-to-Speech API, OpenAI’s ChatGPT 3.5 API, AI tools for the key feature of our project, within the backend. Furthermore, we utilized MongoDB Atlas for our user database, to effectively store and manage sensitive user content.

Challenges & Accomplishments

A big challenge in pursuing the goal was creating an accessible interface for users. With particular demographics in mind, our goal was to create an app that’s easily adaptable to the user and intuitive to use. As a result, we challenged ourselves in pursuing an intuitive UI, with multi-language support. With large text fonts, text-to-speech playback, and visual buttons, our app aims to be as navigatable as possible by all target audiences. By utilizing adaptable APIs and UI/UX design, we developed a globally accessible app.

What’s next for PillPal

PillPal’s architecture is versatile, and could easily be adjusted to meet the needs of various demographics, because of AI’s flexible nature and the exponential growth within the field. We hope to add more accessible features to our app including a microphone to interact with PillPal and expand its capability to answer user’s questions. Furthermore, we hope to provide seamless access to individuals in different scenarios because of PillPal’s flexible design, catering to those who have disabilities and hard situations.