After returning from California where everything was labeled with calories, I was disappointed that I could no longer easily or efficiently keep an eye on what I was eating. Calorie trackers already fill the market, but all of them require the hard labor of manually searching for items or entering calories yourself. With Banana Scana, the dream began with a banana and ended with the ability to easily and efficiently see the calories of every item that you enter. It empowers the user by giving him or her the ability to diet smarter. In a country where obesity is higher than ever, the need for such an application has never been higher.
What it does
Banana Scana is an intelligent application that uses image recognition to label and find calorie information on food items. The user simply takes a photo of the food item he or she is about to eat, and he or she is then presented with a list of options representing what the food item is most likely to be. After selecting the desired automatically generated item, the user's calorie information is then stored in the cloud and in the iOS Health app for easy access. The user can then see all history of food items or access a customized diet recommendation based on the user's personal information. Users can watch the "Today" screen as the day passes and watch how close they are to meeting their calorie intake goal.
How I built it
Banana Scana works by taking an image from the user's iPhone and then storing it in Parse. From Parse, a link to the image is then sent to Clarifai. Clarifai then responds with 20 tag possibilities of what the food item may be.
For retrieving calorie information, we are using over 8,500 food nutritional fact entries created by the USDA. We parsed the data from an Excel sheet, using a Python script, into a JSON object. We then get the 20 tags retrieved from Clarifai and filter them by matching them with a filtering dictionary of food items. This eliminates search time by removing the need to search for all 20 tags in 8,500 items.
Calorie information is then returned to the user, where he or she can select the number of servings. After confirming the food item servings, the calorie information is both updated in the user's Health app and in the cloud using Parse.
Users can then access a "Today" view where they can see how close they are to their suggested goals. The Harris-Benedict BMR formula is used to give each user a customized calorie intake suggestion. Most of the information for calculating this is received from HealthKit. It is then displayed in a user-friendly circular progress view.
Challenges I ran into
Updating objects on the server with Parse, adding user-friendly UI with the sidebar view, using HealthKit when little documentation is yet available, determining the best service for image recognition
Accomplishments that I'm proud of
Team members gained much experience in Objective-C, with one of them going from knowing basics to becoming proficient. We are also very proud of our effective use of image processing and most importantly, the huge impact that an idea like this could have on the health of society.
What I learned
Objective-C, Clarifai API, HealthKit API, Parse API
What's next for Banana Scana
Hopefully an App Store version in the future! In addition, we would like to add monitoring of activity from the Watch and iPhone to even better customize calorie suggestions.