011 - Tally Cally

Count calories by a snap

Comment

Inspiration

Health awareness is often an important topic which is needed to be cultivated to the public. Singapore Health Promotion Board even launch 'Health 365' and several campaigns to promote people performing exercise while able to win some rewards. Using smartphone or any step tracker available in the market, users able to keep track how much activities has been done daily. However, daily calorie consumption is often ignored because it is quite a hassle to manually record down all the meals taken throughout the day. Thus, it defeats the purpose of monitoring individual net calorie intake if only steps are tracked.

We hope to extend what has government implemented, and purpose a simpler way to record daily calorie intake, simply just taking photos via smartphone.

What it does

The system able to recognise the food consumed by individuals simply by uploading a photo, then estimated the amount of calories in that food. Moving on, we hope to integrate the data into 'Health 365' app's existing database.

How we built it

The entire image recognition engine by powered by Convolution Neural Network, a kind of machine learning. We build a webpage as user-interface prototype.

Challenges we ran into

1. In order to train neural network able to recognise food, massive amount of food pictures need to be collected (estimated 1000 photos for food category). We initially objective is to train neural network to recognise every single food items inside a single picture, e.g. for one sushi picture, neural network should be able to recognise every different type of sushi in that picture, then calculate the total amount of calories. In order to that, human need to manually frame and annotate labels to every food items in a single picture. Simply means, for 10 types of food categories, we need to process 10,000 pictures.

We are running out of time, thus, we change our scope to train only ala-carte food, which does not need any label annotation.

2. Using python to host a web is completely a new thing to us. We are extremely exhausted trying to debug the code. Python is used because our food recognition engine is ran entirely in python (via caffe framework). We spend most of our time trying establish communication between webpage to the machine learning server. Lack of experience in networking and html drained our energy away.

Accomplishments that we're proud of

We took some pictures of the food provided by Hack&Rolls sponsors (e.g. cupcakes, pizzas). After training the model for almost 12 hours, we tested the model using Hack&Rolls' food. Guess what, the machine able to recognise red velvet cupcake and pizza. We shout out loud from the bottom of our heart (We can't yell because it is 4am midnight).

What we learned

We are glad that as a team of 4 members, each of us has our own strength. Thus, we are able to complement each other weakness, and get over demonic-24-hour hackathon. Also, we learnt new codes and algorithm, which definitely beneficial to our future career.

What's next for Tally Cally

We wish that Singapore Health Promotion Board or other health awareness parties, entrepreneurs, able to see the great deal in this software. Perhaps, we might get sponsored to continue the project and make it happen.

Built With

Share this project:

Updates