What is it?

A mobile health application that uses vision processing and the Cisco Tropo API to identify medicine based on data regarding the text, shape, and color of the pill, and then coordinates care with your physician, family, caretakers, or others accordingly.

The application could be used by any everyday person to quickly and easily identify pills using one's camera, telling you visually or audibly in case of vision impairment, or can be configured to call a loved one if they forgot to take their medicine, and even their physician if the medicine identified is dangerous or incorrect.

Identifying the Pill

We used a native Android application in order to send a picture of the pill to the server which then used Google's Cloud Vision API in order to read in attributes about the pill and compare the information to our data set.

Coordinating Care

We used the Cisco Tropo API in order to add the ability to theoretically have the app text loved ones if the user has not used the app recently, or if they have received the wrong medicine, identify dangerous medicine, or stop taking the medicine altogether, the application can contact your physician to have them follow up.

There is a lot of room for addition here that we would love to expand on.

Given More Time...

We would love to see this application be able to detect a handful of pills, compare with one's prescription, and coordinate care accordingly. In this way, one could use it to conveniently keep track of and identify they took the correct pills for that day.

We'd also like to better the data sets and visualization by taking in more attributes, test with an elderly population, and collect data on pills causing the most confusion to improve manufacturers' processes.

Accomplishments and Things We Learned

Cisco Tropo API, Google Cloud Vision API, IBM Watson, Android

Share this project: