Pill Identifier

Inspiration

We wanted to develop an app that would help professionals in various careers better do their jobs. For example, our app would presumably help police, who come across prescriptions that could possibly be narcotics frequently, determine whether the drug at hand is a narcotic. On a related note, medical personnel would be able to identify patient prescriptions solely based on the visual appearance of the pill, hopefully reducing the need to run chemical tests to achieve results.

In a closer-to-home application, parents would be able to use our app, if refined enough, to be able to tell whether any medicines taken by family members, including their children, are potential narcotics or addictive/dangerous substances of some sort.

Furthermore, our app would help the elderly, especially the visually impaired, keep track of their prescriptions.

What it does

The web-app takes in an image of a pill, or multiple pills, and indicates the percent probability that it is one of the pills included in the training data set of the machine learning algorithm called Einstein Vision API. For example, if the image of a certain image is dragged and dropped into the corresponding box present on the web-app, Einstein Vision API would process it and provide the percentage probability of the pill being a certain pill used in the training image set, such as Xanax, for example.

How I built it

A web-app was created with Heroku and integrated with Einstein Vision API, which was then trained with a set of 132 images of different pills (15 images of each pill). In addition, some front-end tweaks were performed on the index.scala.html file via HTML and CSS before deployment.

Challenges I ran into

The CSS file used in the web-app deployed from the git repository I forked was closed, and I have limited HTML + CSS experience, so I had to figure out how to override the CSS file specifications with my own. In addition, deploying the app after setting up the back-end with Django was problematic, even though the app itself managed to compile just fine.

Accomplishments that I'm proud of

This was the first time that I had worked with Heroku and Django, both of which I had never worked with before, despite my experience with Python. Though they were somewhat difficult to get accustomed to quickly enough, the challenge in itself was fun and extremely useful, as I will be able to go into next week knowing that I can make web-apps with languages I am already familiar with.

What I learned

I learned that, though there is a lot of information and a lot of languages I have yet to learn, I can really create something amazing if I learn how to integrate what I already know with something I managed to learn over just one night.

What's next for Pill Identifier

Einstein Vision API clearly needs to be trained with a variety of different images that are still consistent enough to hone its predictive power. Such images were not used by us, as they were either unavailable or unknown to us at this time. However, with more images of more medicines, we should be able to improve how well Pill Identifier can determine the type of pill presented, improving health outcomes for the various professionals listed above.