As busy graduate students who like to stay active and in good health, we (Eric, John, Kevin) like to stay active, frequently going running, rock climbing, snowboarding, and cycling together. Coming into this Hackathon, we wanted to develop a tool to help us, and others keep track of our nutritional intake. As graduate students, we wanted to get a wholistic look on our nutritional intake, further than just calories and food groups.
This project really evolved upon learning that google cloud and snapchat were sponsors of the Hackathon. All around us, we everyone is constantly taking photos and videos of their meals, saving them to their google storage posting them to Instagram, snapchat, and Pinterest. Using the available APIs, we could easily tap into these data sets to improve users' health.
UCFood's name was inspired in part by SeeFood from Silicon Valley, but adapted to represent University of California.
What it does
UCFood was envisioned as an application that would allow people to quickly snap a video, take a photo, or describe in speech, what they were eating and get a quick response with the nutritional information of the meal.
The application tracks all the macronutrients and micronutrients that one consumes such as calories, potassium, sodium, etc. This allows for very fine detail tracking of food and ensures that one is not unintentionally consuming more of bad macromolecule such as sodium or cholesterol. Users will be able to see breakdowns of daily, weekly, monthly, and yearly statistics.
This app is intended to be the step tracking equivalent of the food world, and turn eating healthy into a competition amongst friends.
How we built it
The app front-end was developed on swift, utilizing the base sample provided in the learning tutorial. Photos are taken on the iPhone and sent to a flask back-end hosted on a google app engine. When the media is sent via HTTP to the flask back-end, python code runs the necessary code to pull information from the video (from audio and visual). This feeds out a list of items that are likely to be consumed, such as "chicken caesar salad and a glass of orange juice". This is then further reduced into its constituents using google natural language processing, yielding "chicken caesar salad" and "orange juice". Using queries to USDA, nutritionix, and edamam, the nutrition information for the meal is then pulled up. All the components for the the meal saved to a mongodb backend along with all the nutritional information.
One of the main hurdles encountered during the development of UCFood was the connection and data transfer between the front-end iPhone application and the backend. We ran up against bandwidth limitations when sending videos to the flask back-end, but it was a very good learning experience for getting over challenges. Additionally, we had to think about data structure and user data storage.
As materials science and biochemistry students, who have taken few computer science courses, we have learned a great deal about integrating applications. Having only used python to analyze basic experimental data in the past, we were able to learn a great deal about using APIs to accomplish more challenging tasks such as natural language processing, machine learning, and database storage.
Future Work on UCFood
UCFood, while functional is not fully featured and we will be working on this app more in the future. Mainly, we will be working on implementing a gamification system of meeting daily and weekly intake goals, rewarding users with badges for maintaining healthy eating habits. Additionally, we will be using visual tool kits to estimate quantity of food to be consumed. Furthermore, we did not have time to setup integration to automatically pull food related photos from linked google photos and snapchat. The graphics of the application are in much need of a clean, professional design. The largest stretch goal for us is to use ML to determine components of recipes and build a database of nutrition facts for home cooked meals.