Bitesize

• 

  Prototype app that is able to identify and provide calorie/macro breakdown for over 200 foods. Runs off a CNN neural network I trained

• 

  Home Page UI

• 

  Camera UI

• 

  Deep-learning analysis provides you with exact food items present, and their calorie/macronutrient breakdown

• 

  Weight Progress chart

Inspiration

In order to prevent complications such as risk of obesity, heart disease, or to simply lose weight, many Americans are interested in tracking what they eat. However, most current mobile apps such as MyFitnessPal and LoseIt require manual data entry, which is tedious and time consuming. Consequently, most users do not use such apps for very long. Furthermore, amateur self-reports of calorie intake typically have an error rate that exceeds 400 calories per day.

A second similar, problem occurs when individuals hire a dietician to plan out their diet for them. The problem here arises because dieticians do not have the ability to see every meal that their client eats. Clients convey their daily food consumption to their dietician in vague descriptions, which many professionals such as Chelsey Amer claim tend to have worse error rates than if the client had self-tracked their calorie consumption on their own.

My solution to the problems of these two separate consumer markets is the founding of Bitesize, a start-up focused on developing two products. First, a retail mobile app that uses sophisticated deep learning algorithms to analyze a still photo of food, and estimate how many calories are on the plate. This will be a system which can recognize the contents of your meal from a single image, and then predict its nutritional contents, such as calories, and will be aimed at American adults who are currently in the MyFitnessPal userbase. Second, Bitesize will develop purchasable APIs that use the same AI algorithms as the mobile app, however these APIs can be custom implemented into American hospital’s existing personal websites. From the comfort of their own hospital’s websites, the APIs will allow dieticians to get a clear understanding of what their clients have been eating by viewing both pictures of their meals, and the calorie/macronutrient breakdown.

Finally, using MyFitnessPal as benchmark, and assuming we have a user base that is even 1/10 the size of theirs we should at the bare minimum be able to gather between 10-50M total users for the retail mobile app, and given the amount of dietician facilities in the US, we should theoretically assume around 6000 potential consumers to use the purchasable APIs.

What it does

For a very simplified overview, the simplest version of the deep-learning algorithm assumes that the user is eating at a restaurant for which we know the menu. In this case, we can collect images offline to train a multi-label classifier. At run time, we apply the classifier (running on your phone) to predict which foods are present in your meal, and we lookup the corresponding nutritional facts. We apply this method to a new dataset of images from different restaurants, using a CNN-based classifier, significantly outperforming previous work. The more challenging setting works outside of restaurants. In this case, we need to estimate the size of the foods, as well as their labels. This requires solving segmentation and depth / volume estimation from a single image. Essentially, Bitesize presents a CNN-based approach to calorie and macronutrient counting. The long-term goal for this technology is more wide-reaching, as once people will start using Bitesize, we’ll collect data, and it’ll get better over time as we will continue to train the model with the vast amounts of available caloric data users will provide as they use the app.

How we built it

Built a mock UI, and created an MVP version of what the actual AI algorithm would work like using Convolutional Neural Networks, js, wolfram alpha, python, and jupyter notebooks.

Challenges I ran into

Definitely the amount of time given during the Hackathon.

Accomplishments that I'm proud of

Creating what I believe will be "the killer food app," given in concept this would be incredibly popular. Also training a prototype app that has the ability to detect over 200 different foods, from data provided by Kaggle, and provide their calorie breakdown.

What we learned

I learned more about convolutional neural networks and how to pitch a startup idea.

What's next for Bitesize

Hopefully the creation of a genuine company

Built With

• ai
• deep-learning
• javascript
• jupyter-notebooks
• neural-networks
• python