NutriFit

Inspiration

One common problem that we all agreed on was the difficulty of choosing what to eat, or how healthy to eat? It can be tricky to track down the nutrition facts of a recipe. Our goal is to smoothen the path to a healthier and easier lifestyle.

What it does

NutriFit excels in image recognition. It's main two features include a receipt scanner and dish recognition. Using these two, Nutrifit is able to collect ingredient data and generate probable recipies. It is also able to scan and recognize dishes and output an estimate of the levels of calories, carbs, and proteins a dish has.

How we built it

Nutrifit uses tesseract to be able to effectively scan receipts. To input the image to tesseract, we use PILLOW to grayscale it and remove black spots, borders, shadows, etc. It is then fed into tesseract, which will read the gray scaled image and return the text. Finally, it is plugged into the OpenAI API in order to output a list of ingredients that the receipt used.

Challenges we ran into

Some challenges that we ran into was integrating frontend and backend. Additionally, it was also very difficult to get tesseract to return an accurate text evaluation. In the end, it was more than 90% accurate, and after plugging it into the OpenAI API, became virtually fullproof.

Accomplishments that we're proud of

In order to get an accurate estimate of calorie and other nutrient information, we originally tried to use another library, API, or data set. In the end, it ended up being Fairly, accurate given that the only input was an image. To achieve this, we used a feature of our product, which is recipe generation. In one command, we let the OpenAI API determine a feasible recipe and ingredient amounts for the image shown. Following this, we were able to get it to determine the nutrition facts for each individual ingredient, sum them all up and get a very accurate estimation. We compared it to items from stores such as the Cheesecake factory, as well as other stock photos from google. We were all really impressed by the accuracy, and were proud of this as it demonstrated an effective use of the application.

What we learned

We learned how to integrate frontend and backend, by using a database that connects the two and transfers data effectively. We also learned how to use Pytesseract, and improve our knowledge with PILLOW.

What's next for NutriFit

Next for nutrifit, we would like to have the possibility to input more data in order to increase the accuracy, if possible. If the name of the dish was given, it would make sense that the model would make a better prediction. Likewise, it would be cool to integrate the recipt scanning with specific grocery companies to smoothen the process, save time, and reduce the chance of incorrect ingredients and recipes that are not cookable.