Foodie@7830

Inspiration

What's a decision that you have to make every single day of your life? What to eat.

Every day, we have to decide what to eat. Every time we hang out with our friends and ask them what they would like to eat, we will get answers like 'Whatever' or 'Up to you'. While this may seem like a trivial problem, we might waste hours scrolling on TikTok and Instagram, hoping to find a place to eat.

This inspired us to create an app that helps users decide what to eat, so that they have one less decision to worry about today.

What it does

Our app offers personalized restaurant recommendations using AI. Users input their preferences, such as cuisine type, dietary restrictions (like vegan or gluten-free), and budget, and the app suggests suitable dining options. The app can process real-time queries like "Help me find a Japanese restaurant nearby" to suggest meals based on the user's current location and preferences.

How we built it

Ideate: We did start with just a simple idea of building a supervised learning model to predict what are the best options based on user preferences. However, after great development and researching through papers about recommendation systems and being inspired by commercial foodie apps, we significantly improved our app, from leveraging multiple Deep Learning (NLP) models to improving user experience with our software (UI/UX, authentication, voice recording, ...)
Data Collection, Processing, and EDA: We collected our data from multiple channels, such as public data from Kaggle, HuggingFace, data scraping from Uber, and API fetching from Google API as well as generated dummy data to train our recommendation engine. We took long time to turn the meaningless strings of comments or food cuisines, types to model understandable by leveraging pandas, Bert model, and Google word2vec model.
Model Training: We fine-tuned a Named Entity Recognition (NER) model using Hugging Face’s transformers library to accurately extract entities such as cuisine type, food preferences, and budget from user inputs. We used another NLP model to compare the food type compatibility between user input and restaurant data. We also retrain our main recommending system using reinforcement learning approach, from user ratings our our recommendations.
Recommendation Engine: We developed a recommendation system that uses the processed data to provide personalized suggestions. The system considers factors like the weighted rating of a restaurant, distance from the user, the type of food it serves, and the price range. We retrained our model for better performance by using two approaches in recommending system: collaborative and content-based filtering. With these two algorithms, we can cluster users, restaurants, and dishes to provide better insights and recommendations.
Integration: We integrated the AI models with the recommendation engine onto our appealingly designed website (mobile concentrated) to handle dynamic user queries and provide suggestions in real time.

Challenges we ran into

"Where do we start?" We had an idea for an app. But there were so many things to do that we didn't know where to start, what feature to build first, and what the workflow should look like. We had to define the problem ourselves and then went about solving it step-by-step. Along the way, we also needed to learn different algorithms and models that we didn't have any experience using before. Furthermore, we were surprised that there had not been any food recommendation systems on social platforms like HuggingFace or commercial apps. Therefore, we were scared but excited to try out and implement new ideas along the way.
Data Labeling and Preprocessing: Creating a high-quality labelled dataset for training the NLP models and testing the recommendation system was challenging. As we mentioned before, deep learning models specialized for food have not been researched. First, we had to ensure that the dataset was comprehensive and covered a wide range of possible user queries and preferences. Second, we had to fine-tune our Ner and Bert model to effectively retrieve our wanted features from the meaningless strings like user comments. Finally, we had to use the word2vec and use cosine similarity technique to create meaningful labels.
Real-Time Processing: Implementing real-time processing of user inputs and integrating it seamlessly with the recommendation engine posed technical challenges, particularly in ensuring fast and accurate responses.

Accomplishments that we're proud of

We successfully built an app (from scratch!) that provides highly personalized restaurant recommendations based on user preferences, which may be a fascinating idea for a startup commercial project. Our app can understand and respond to user queries in human language. We wrote the logic and algorithm for the recommender system ourselves. We also aced by creating a completed software architecture, visualizing the interactions between users and the system, and the data flow from user input, preprocessing, calculation, prediction, database saving, and finally retraining.

What we learned

We learned the end-to-end process of building an AI-powered app that has practical use in our daily lives. But more importantly, we learned how to learn. All members of our team had to learn new skills, whether it was scraping data from the web, writing the algorithm for a recommender system, or fine-tuning an NLP model. Most importantly, we formed a friendship and everyone enjoyed meetings, discussions, and software developments with each other.

What's next for Foodie@7830

We plan to enhance our app by incorporating additional features such as allowing users to scan the menu of a restaurant and getting recommendations for particular dishes that suit their taste and health goals. We have also written an algorithm to implement a simplified reinforcement learning-based recommender system to improve recommendations with user feedback. We also planed to turn this to mobile software for a better user experience using Flutter and created better algorithms.

Note: Please contact us for a trial experience for our app. We will share through Ngrok.