Carb Genius

Inspiration

Managing Type 1 Diabetes is a 24/7 battle that every Type 1 diabetic must wage daily to control their blood sugar levels. People with type 1 diabetes (T1D) must balance the number of carbohydrates (or “carbs”) they consume with the right dose of insulin. That’s why counting carbs in any food item they consume is so important. This is especially difficult while eating in restaurants as the number of carbs in certain food items can vastly differ from restaurant to restaurant and searching for the nutrition information on the restaurant’s website can be a challenge given the different layouts of each restaurant’s website. As a parent of a Type 1 Diabetic kid I know firsthand how difficult it is to assess the number of carbs my kid is eating when we are at a restaurant. Oftentimes we just make an educated guess which leads to bad insulin doses which in turn lead to high blood sugar levels resulting in short and long term effects.

What it does

That’s why we decided to build a smartphone app using which Diabetics can scan their food with their smartphone’s camera and get the carbs info instantaneously which in turn allows them to take the appropriate amount of insulin. This app not only relies on visual clues to figure out the food item but also uses location details and user habits to figure out the restaurant and search the restaurant’s specific nutrition details quickly.

How we built it

As an MVP we decided to build this as python flask app to work for three restaurants namely, McDonalds, Wendy’s and Taco Bell. We worked on developing a separate custom image recognition model using Tensorflow for each of these three restaurants Currently the app is deployed in AWS West 2 region and AWS Wavelength Zone in SFO

Challenges we ran into

The biggest challenge we ran into was testing this app on the wavelength zone. We couldn't quite connect to the app on the wavelength zone from outside the SFO area. We also realized how difficult it is to build an image recognition model. Build a model to tell the difference between a burger and a taco is easy but it's very difficult to make the model tell the difference between a Big Mac and a Quarter Pounder with Cheese (my favorite at McDonalds)

Accomplishments that we're proud of

We were able to successfully deploy the app using python flask in AWS West region. We were also able to build the tensorflow models for each restaurant which were 60% accurate.

What we learned

We learnt quite a bit on building image recognition model and building flask apps. Build apps with flask is a quick and easy way to get up and running with ML/AI apps.

What's next for Carb Genius

As next steps we plan to merge this data with the type1 diabetics Continuous Glucose Monitor or CGM and provide insights into how different restaurant foods affect the person’s BG levels. This is where the real value is for Type 1 Diabetics.