The patient uses their phone camera to take a picture of their prescription pill bottle. Then, they send that image to our Twilio phone number through SMS or MMS. Afterwards, Twilio API transfers the image to our Flask application which uses Cloud Vision API to parse the picture for dosage information. Finally, the app sends text notifications to the patient reminding them to take medicine when it is time.
We wanted to create something to improve patient health and we also wanted to learn how to use Flask and other APIs. We were inspired by NIH's 2011 paper, "WHO Cares?," showing there are over 50 percent of patients who do not take their prescription medications.
Some challenges included getting the APIs to connect, learning about Heroku for web hosting, and properly parsing text.