We were inspired by Dr. Ziegelstein's lecture to focus on giving patients the best bang for their buck, especially as health care prices soar. Whenever pills are mixed or taken out of their container, identifying them can be a big problem; if you don't know which drugs a pill contains, you should never take it. Many people with this problem can simply use online pill identification software by inputting the imprint, colour, and other details about the pill, but this isn't always the case for elderly patients or others with poor eyesight who may not be able to read the imprint clearly, leading to drug misidentification. This disadvantages those in this position and increases the practical cost of their healthcare through wasted drugs.

What it does

In order to mitigate unreliable eyesight -- either misreading the imprint or colourblind patients misidentifying the pill colours -- we sought to develop an app which allows users to take a picture of the unknown pill and use AI to correctly identify the pill for them and provide them with crucial information about the drug.

How we built it

We used different computer vision API's to get both the drug imprint (using Google Cloud's OCR API) and colour (using Clarifai's colour API) to determine the details of the drug. After integrating these API's into the web app, it was a straight shot to the pill identifier to input the pill information and ascertain its identity, finally outputting the drug's name and details.

Challenges we ran into

Originally we wanted to build the app as an iOS app in Swift but we found that the API's we wanted to use weren't compatible with Swift. After that, we swiftly shifted tracks to build PillPix as a web app.

What we learned

We learned how to use some frameworks that we didn't know before as well as strengthened our existing skills in the languages we did know.

What's next for PillPix

Hopefully, those who cannot identify a pill and do not have the ability to reliably read its imprint or determine its colour will be able to use PillPix to safely identify pills.

Share this project: