-
Main Page
-
Pop Up with Info about Pill
Inspiration
The inspiration was my grandparents who often had to take lots of pills, as a part of their daily routine. They would often ask what these pills were, and I would always struggle to answer. With this project that problem will be no more, and I can be confident when they take their dosage(s).
What it does
By taking a picture of the miscellaneous pill on both sides, it can be matched to a database which using Image Processing can help identify what the medication is most likely being shown. This can help with not mislabeling pills, which cause over 6000+ fatalities in the US annually CDC.
How we built it
Using an API that was scrapped from a PHP webpage, we were able to have a database which can be compared for our images to be cleaned, and built identification markings, sizes, and colors which are used to figure out which pill is being identified.
Challenges we ran into
The biggest hurdle was the UI to AI models conversion through native libraries. Due to python not being able to be run in Swift or Java, .so files were required which were called using Native Modules. In the case of Java, we were use Java Native Interface (JNI) and converting the Python code into a Rust based backend for "ease" of implementation
Accomplishments that we're proud of
I am most proud of having a functional image processing model/system which is extremely accurate. I'm also proud of the main UI page with the background being a live feed from the camera (as I was inspired by the Google Lens app).
What we learned
I learned the difficulty for cleaning images, to get solid black and white processing. I also learned of the difficulty of building a working UI and converting python to native libraries for the front-end. I also learned that overestimation of time is very prevalent when working, as we downplay the difficulties in the project.
What's next for pillAI
The next challenge is to officially move the Python testing scripts to Rust based scripts for better memory usage and easier implementation into C for JNI and iOS native modules.
Log in or sign up for Devpost to join the conversation.