While surfing on Instagram or any website, there are pictures of food where you cannot find the restaurant. An awkward and long-time-no-talk acquaintance posted a food picture without tagging the location of the restaurant, or the picture has surfed the web so long that there is no citation anymore even if it came from a store. The store could have earned more customers if it was known, and most importantly, you are craving for it. We thriving to help the food-lovers from around the world to their most immediate cravings by accurately identifying the dish in social media posts and suggesting relevant nearby restaurants.
What it does
Foodie Lens serves food-lovers from around the world to their most immediate cravings by accurately identifying the dish in social media posts and suggesting relevant nearby restaurants Utilizing the internet food culture to raise presence to local businesses
How we built it
so our project consists of 3 parts: machine learning where we classify the images, the rest API where we get the images from the users and feed them to our model and get predictions and finally the google maps API which we use to get the nearest restaurants that provide the dishes in the images.
For the Machine Learning Part
we used Fast.ai library with PyTorch and used pre-trained models to train a food classifier. we used transfer learning for faster and more accurate results.
For the REST API
we used python with starlette to create our back end which communicates between the interface and the model. and we used docker and render to deploy the model.
For the Google Maps API
Challenges we ran into
The dataset was really big and the classes were not balanced so at the beginning of the work we made a smaller version of the dataset so we experiment faster and we also removed the class with a very low number of images compared to the rest of the classes. during the last day and after finishing most of the work like the REST API and the Google Maps API we trained our model using the full dataset.
We found some challenged at the beginning with web development as all of the 3 group members didn't much experience with it but after that, we found a 4th team member and he helped with the web development part.
Accomplishments that we're proud of
we are mainly proud of three things: the first is being able to work the google maps API with all the limitations. And the second one is getting a good accuracy on the dataset that we used. and finally, we are proud that the integration between the different parts of the project worked together and as we wanted it to be.
What we learned
we learned about new things like machine learning and how these model learn, we also learned about web development and the deployment of machine learning models using REST APIs.
What's next for Foodie Lens
Scalable to most social media platforms (Twitter, Instagram, Facebook, Reddit, etc.) Integrate with Uber Eats when more detailed documentation are accessible Since 88% of active users of Instagram are outside of the US, will focus on expanding to the global market