To Help people know nutrients in their food.
What it does
Speak Nutrients in food
How we built it
Created a compute engine on GCP of Linux. (We used Ubuntu as it supported python by default).
Scrape High-Quality Images from google search filtering for better efficiency of the model being created. In our case, we achieved an accuracy of 80.7%.
The model we created is based on CNN (Convolutional Neural Network) like google’s Inception-v3 model on GPU instance. The model used was for Image classification with 40 diverse food items, with about 100 images for each category with 10% of them as the training set with 22 levels with 8000 set as maximum training cycles.
Imported the model files generated by Tensorflow on GCP into an Android application using Tensorflow SDK. Used NDK for C++ code base for detection which computes faster than Java code thus enabling lag-free calculation using GPU supports.
The images were converted to Numpy array of size 299 x 299 from 720p video frames captured from phone’s camera. Deciding on a threshold of 51% thus not having confusion among two categories by applying minimum distance required for decision set as 10%.
Used TTS (Text To Speech) service for letting the user know what food item has been detected. JNI code base used extensively for such purposes.
Locale Support for different languages has been provided by using Google’s Translate API over GCP for initialization. Hence, providing non-network dependent machine learned application.
Challenges we ran into
Not being able to sleep 3 days doesn't seem much when your code keeps crashing. And when you know that it is because of a corrupt file which you are parsing for Machine Learning, there is not much you can do than just shake your head.
Accomplishments that we're proud of
I'm proud of not attempting to incorporate any of the sponsor companies' APIs just to be eligible for their prizes because that would have caused unnecessary complexity and distracted me from achieving the main goal.
What we learned
We are graduate students of cybersecurity. Using tensorflow was completely new to us, but it is really simple with the great documentation written for it. We were never going to leave until we complete this project.
What's next for SenseFood
We are going to take this wonderful fully functional project way more ahead.