I was looking at a bag of chips and wondering what it really had in it. I glanced at the ingredients and realized there was no way I could comprehend what each word meant, and the ingredients looked completely foreign. I understood that there was a need for a translator that would provide much more information about these ingredients
What it does
When one takes a picture of an ingredient list on the ClearNutrition website, after cropping the ingredients, our program will search the web for nutritional information on each of the ingredients present. Additionally, our program will search the web as well as a local database to find out if the ingredients present are linked to any common health diseases or conditions, then formatting this data and printing it on the website.
How we built it
We built the back end using python. We used opencv to create a camera stream from which a picture could be taken. We implemented pygame to create a user-friendly interactive cropping system for the users to crop the image. From there, we used Microsoft Azure's Computer Vision OCR API to convert the text into a string which we then parsed through to find each ingredient separately. From that point, we utilized BeautifulSoup to create a web-scraper which searched various websites including BBC and wikipedia for data on each ingredient, then returning formatted and cleaned-up data. From there, we used django to connect the back-end to the front-end which was built in HTML5.
Challenges we ran into
We were all new to some of the libraries we used. Additionally, for our software which checked for ingredients that link to common health diseases, we ran into much trouble finding a database which would allow us to do so in any way. This lead to us re-implementing an aspect of our web-scraper for this operation and then implementing a local database with the most common ingredients in packaged goods and their links to various health diseases and conditions
Accomplishments that we're proud of
Creating a user-friendly front end. Utilizing Microsoft Azure's API Utilizing OpenCV and PyGame for a user-friendly picture & cropping system Creating a nicely functioning webscraper with BeautifulSoup
What we learned
We learned a lot about each of the libraries which we used in our implementation. We also struggled through some HTML concepts but as we advanced we found ourselves more comfortable with the language as a whole.