PS: Please watch the demo video with closed caption (subtitles on), I've explained it there.
Inspiration
A lot of times, we come across new dishes and wonder what kind of food that is. Well, this app will be there to help you up. Just point your camera towards the food and it will tell what kind of food it is instantly. And yeah, this app works offline.
What it does
It has a model trained to recognize food categories and uses your phone's camera to recognize the food that your camera is seeing by passing the frames through the model.
How We built it
I (Sayan Kundu) made the application using Flutter framework on the Dart language. My Teammate Triparna Ganguly trained the model using TensorFlow framework on the Python language. She trained her own Convolutional Neural Network from the InceptionResnetV2 using the Food-11 Dataset; having done all of the steps, she converted the model to TensorFlow Lite. I just implemented the TFlite model into the application, making it work with the camera.
Challenges We ran into
Well, the first challenge was to find the sweet spot between the camera API and the TFLite API. And one thing I learned from this challenge is this: "Never ever go for the highest setting". Another challenge we faced was to get the accuracy of the model to be high enough for it to work on real-world applications.
Accomplishments that We're proud of
Maybe that we made it happen.
What We learned
Triparna - I learned to trust my abilities and capabilities. Patience and hope were the additional keys.
Sayan - I basically got a spark of interest in Machine Learning now. The way this stuff works amazed me.
What's next for WhatFoodIsIT
Future plans include
- Adding support for more categories, like even identifying what food it is
- Making a better UI (this current one is just a barebone)
- Optimizing performance
- Probably even add an option to get a recipe to make that food.
Log in or sign up for Devpost to join the conversation.