Recipe recognition and healthy replacement recommender

Screenshot of text recognition
Main screen
"Screenshot" of part 2

Inspiration

We're all trying to find healthier food and want an easy way to calculate caloric and nutritional information from written recipes (like in recipe books!).

What it does/How we built it

It uses OCR (Google Vision API) to scan a recipe and it extracts the needed information to calculate the nutritional information. It then queries the USDA Food Composition API to return the calories, fat, carbohydrates, protein, and other information. From there, it stores it in a local database on the phone. It then will find healthier replacements for that food, if the user desires. The recommender uses Python and integer programming to find optimal replacements.

Challenges we ran into

The challenge was communicating between the phone and our Python recommender API. Additionally, we experienced challenges learning Google's Vision API for Android, particularly its high failure rate for static images.

Accomplishments that we're proud of

The text detection works on photos of recipes and it correctly calculates the nutritional information. Created a visually appealing interface for food recommendation using integer programming.