During a conversation with a friend, we realized how hard was to keep track of your diet. Right now, the vast majority of apps to track what do you eat require a lot of effort from the user: manually input product name, expect the app to know what is it, then try to estimate how much did you take from that. It can be really difficult to keep it, requires a really strong will.
Following that problem, I decided to try to build a bot that minimizes the number of interactions required to get a result.
What it does
EatTracker is an almost public messenger bot that recognizes tracks the calories you take. It does it by scanning food from the pictures you send him and approximate quantity you send him. EatTracker gives you an immediate approximate number of calories. In addition, it also offers nice dashboard to check your eating history, together with a (for the moment) small amount of statistics.
How I built it
The whole backend runs on Django. For the image recognition process, I used Clarifai Food pre-trained neural network, through its REST API. To extract average nutrition values, I used Wolfram Alpha results API. In order to build the bot, I set up a Messenger bot plugged to the backend previously mentioned.
Challenges I ran into
Probably the most challenging part was to use the predicted data and improve it with user input, minimizing the user efforts. Also, my initial idea was to build it on my own partial-trained convolutional neural network. However, due to my laptop having low resources (4Gb of RAM) and taking forever to install tensor_flow source library, I decided to avoid doing it on my own, as I wasn't certain that it would be ready on time. In addition, my teammates, who knew more than me regarding neural networks, disappeared, which made me realize I had to reduce the scope of my project.
Accomplishments that I'm proud of
Some of my favorite accomplishment is probably the "interface"/"interaction". I really think that its fast process helps the user in a great way.
What I learned
I've never programmed a bot before, especially a Messenger bot. I was really looking forward to do it and I'm glad I did it. Also, even though I couldn't apply it to this project, I learned the basics of a neural network. Specially I learned a bit more about convolutional networks and how to do knowledge transfer process retraining the last layer for a different feature recognition (my plan was to do this with the 101_Food public dataset and Google's pre-trained Inception v3 network).
What's next for EatTrack
- Train the neural network using user's live feedback to improve the classification
- Use a self-hosted neural network
- Add statistics to user dashboard
- Use user data to provide feedback on diet and improve the user diet
- Predict future diet distortions in user diet
- Add gamification to improve user engagement
- Add product classification (vegetable, meat, fruit...) in order to study user's habits
- Create models of eating habits for users (hourly...)
- MAKE THE BOT PUBLIC ONCE FACEBOOK ACCEPTS ME
I guess there would be even more ways to continue with EatTrack, but that's all I got in my head right now.
Logo: Tracking Location by Friedrich Santana from the Noun Project
Also: bought eattrack.tech but it's not a good idea to change domain of a bot at this hours :D