Ever wondered all the impulsive food shopping we do in stores like Walmart or WholeFoods, thinking "Oh, I will eat the chips while watching Netflix", "I will eat non-fat yogurt because its healthy " but after a while even this chain of thought breaks and you just shop. But what if you are just a picture away from self-realizing which all items are really healthy. What if you don't have to bother about checking Nutrients content of each item and instead you get a beautiful visualization of the nutrient content of all the items you have shopped! Yes, just click a photo of your bill and NutriCheck can tell all the nutrient content of the food items bought by you.
What it does
NutriCheck tells its users the nutrient content of each food item purchased. Also, it can tell which food item has the fattest content. Given a purchase bill, NutiCheck converts the image to text and gets the nutrients values from government website US Department of Agriculture - Food and Nutrition (USDA). The data is then presented pictorially in the NutriCheck.
How we built it
We are using the concept of the Universal Product Code (UPC) to track the nutrients of the items purchased. The bills on the Walmart have WUPC and we are extracting UPC from them using Walmarts API. This UPC is then fed into USDA API, which gives the nutrition values. The data is then cleaned and demonstrated on NutriCheck .
We are using a couple of APIs and integrating them to build NutriCheck. These are:
- We are using Google's cloud vision API to convert the image to text. Walmart bills have MUPC code, these codes are converted to UPC and then the list of valid UPCs is exposed to UI.
- UI uses the US Department of Agriculture - Food and Nutrition (USDA) API to fetch the nutrients value of the food items.
- Using D3.JS nutrient data visualization is done.
Challenges we ran into
- Parsing the shopping bills, they are not at all informative and extracting information like Universal Product Code from the same was really challenging.
- Converting Stores specific product IDs to Universal IDs.
- Integrating all the vast variety of APIs into one system was a huge task. language integration and data parsing.
- Doing all this in less than 12 hours.
Accomplishments that we're proud of
There are various applications which tell you the nutrient content of individual food item, but collective information is not present in the market. Coming up with such an idea and successfully building a prototype was a huge accomplishment.
Integrating APIs from Google Cloud Vision, from government sites and coming up with a codebase which works end-to-end.
Successfully hosting the site on GCP.
Building a nice UI experience.
What we learned
- Setting up GCP.
- API integration.
- How to use OCR data
What's next for NutriCheck
Currently, we are only scanning Walmart bills, we plan to do that across all the stores and we plan to improve the visualization which captures more informative data and which can tell better and healthier options for the items in the bag.