After speaking to the Express Scripts representatives, we discovered that identifying pills is an issue in their industry. With so many generic brands creating common medicines without unique forms, medications out of their containers can be difficult to distinguish. Our goal was to try to attack this issue with an easy to use practical mobile application that physicians, pharmacists, and customers can benefit from.

What it does

Our mobile application requires the user to take a close snapshot of an unidentifiable pill which is then fed to the neural network and is compared to a database. In its 1.0 version it will be using the photos that consumers snap, as well as photos from the NIH database, to train the AI network. In later iterations, the training will be lessened as the AI will be able to identify any prescription just from a photo with a high amount of certainty. The application has been built with the long term vision in mind, and hopefully it can create the largest prescription pill photo database in the world.

How we built it

The front end is an android studio application, while the back end is a python driven server found in the Google Cloud Platform. The python driven server is a supervised Tensorflow neural network which is fed a small sample of classes for the proof of concept.

Challenges we ran into

We ran into quite a few challenges along the way. One path we tried to pursue involved using Google's Cloud Vision to identify the pills imprint, shape, and color, but realized after implementation, that pictures taken of the embedded imprint which is common to many pills, were not distinguishable. To solve this issue, we used Tensorflow to put the photo through a series of filtering like gamma and contrast changes which makes the imprint more accurately recognizable for identification.

Accomplishments that we're proud of

We're proud of getting the supervised neural network to function because that was a new experience for all of us. We're also proud of each other for tackling a problem none of us had much experience with. We started with a vague idea, brainstormed together, divided the work, and helped each other a surprising amount. Overall we had a very positive experience and we're happy to have made new friends.

What we learned

We learned that teamwork plays a huge role in development. Along the way, we learned to identify prescription pills, how much to appreciate a well made database, and we learned a whole ton about machine and deep learning.

What's next for SnapRx

You tell us, if you're interested, we're willing to continue the journey to make this the most successful pill database on the market.

Built With

Share this project: