We were tired and frustrated with endless menus, agonizing choices, and the regrets of meals past. All too often, diners end up ordering dishes that simply aren't good, or ordering dishes that are simply popular on Yelp. Using the power of code, we were determined to develop a smart tool that utilized cutting-edge technology to provide a location-based, document-scanning, hunger-vanquishing mobile app.
What it does
Foodi simplifies ordering and dining for users.
Foodi scans a menu using a mobile phone's camera function, then uploads the image to the cloud for processing, including optical character recognition. Google Cloud's Vision API provided the ML model for converting image data into text. Through an HTTP request that triggers our backend Google Cloud functions, Foodi uses a web scraper to find reviews from previous diners with a fuzzy search and returns snippets from reviews mentioning the dish of interest. Moreover, Foodi runs contexts and mentions in the Google Cloud Natural Language sentiment analyzer, returning an automatically generated rating for dishes. Finally, an in-built web-scraped culinary dictionary rounds out the user's understanding of a selected dish. Foodi gives the user a consummate understanding of each and every dish, turning diners into critics.
How I built it
Foodi was built on a React Native Expo front-end and a Google Cloud server-less cloud backend.
We utilized a variety of Google services such as Cloud Storage, the Vision API, the Natural Language API to perform the database, OCR, and sentiment analysis functions. We also used Cheerio web scrapers to get the culinary dictionary information and the previous reviews. We deployed cloud functions written in node.js to the Google Cloud Platform, which performed much of the backend processing.
Challenges I ran into
Foodi is work borne from sleep lost, tears shed, and blood spilt. We endured hours and hours of debugging, and struggled with smooth integration between all the moving parts, from database to the many APIs to the front end. For all of the team members, it was our first team using any of the Google Cloud services. And since we were using the free trial of Google Cloud Platform, we also ran into quota limits on our API calls. But ultimately, our struggles have created Foodi!
Accomplishments that I'm proud of
We think the app speaks for itself as an accomplishment! We bit off a little more than we could chew, which is an issue that Foodi would solve for any dieting diners. We’re proud of ourselves for overcoming our inexperience with Google Cloud and the multiple APIs (that each came with their own problems) and delivering an app with so much functionality and utility in less than 24 hours.
What I learned
We learned a great deal of the utility that the Google Cloud Platform provides, and we got to learn a lot more about information pipelines and effective database use and management. We also deepened our little previous knowledge of React Native and mobile web dev.
What's next for Foodi
We hope to further develop the app with features such as automatic menu segmentation using Document analysis and a cleaner and faster interface. We can also sort the reviews based on other factors (like length, seniority of reviewer) on top of the sentiment score to provide the most comprehensive snapshot about a dish. We also hope to employ a realtime AR aspect of our app that can provide virtual annotations to menu items. There's great potential for further development and improvement!