For international travelers, eating out at restaurants is often much more difficult than it has to be.
For example, take this scenario: you find yourself at a traditional, famous restaurant in Seoul and just as you finally get seated to order, you realize: How can you order off a menu when you can’t even read it? Of course, studying the language beforehand could help, but even so, international foods bring a widely different variety of flavors, ingredients, and food preparation that aren’t obvious in the name itself and critical to know for those with dietary restrictions, medical conditions, and taste preferences.
To overcome this, D’éclaire does something that restaurants can’t: providing a menu that is not only readable in every language - but also contains key information to guide customers to the highest level of satisfaction. Traveling to different countries is a culture shock for tourists, and in particular, the language barrier can be a major obstacle. To alleviate this, we developed a web app, called d’éclaire, that translates a menu in a foreign language into any language you like.
What it does
Not only does this app provide you with textual information to decide on your order more easily, it also web scrapes the images of the dishes to give you a visual representation of the food that you may want to order.
How we built it
D'eclair allows you to upload an image of a menu that you would like to translate. After uploading an image, d’eclaire uses the Google Vision API to perform text detection in order to extract each of the menu items and their corresponding prices. After extracting the menu items, users can select their target language (the language they want the menu to be translated into). Then the Google Translation API automatically generates the translated term onto the next page. Each original menu item and its translation are rendered out onto the page. In addition, we utilized Beautiful Soup to web scrape each of the menu items and create images of the foods so that the user now has a visual representation of the food that they want to order.
Challenges we ran into
First, we spent a lot of time brainstorming an idea that would be useful, unique, and implementable- dropping many great ideas in the process. We finally decided on D’éclaire due to its large target audience and unique solution to a problem we have not yet seen tackled (and as serious foodies, we were all particularly passionate about this app). When picking the technology to use, we wanted to try learning React but had to use Django instead because of the difficulty incorporating the Google Vision API in React. Lastly, we used many Google API’s for the first time, and struggled with getting the flashcard interface to connect with the object detection in the backend. We also found it difficult to parse the texts detected by the API to the correct format to use it on the frontend.
Accomplishments that we're proud of
We are proud of integrating various Google APIs into a functioning, highly-useful app. Connecting the various pages together and displaying relevant information from the backend into the final, translated menu proved difficult but was ultimately successful. For this hackathon, we also utilized new functionalities such as web-scraping for images based on a keyword and created the card-like features on the front-end with little prior experience. We believe our app accomplishes our set intention fully and also provides high value for its users.
What we learned
We learned a lot about Google’s Vision API and Google Translation API, web scraping pictures, and importing local excel data. For the backend, we also became more familiar with using Django; for the frontend, we got to practice using Bootstrap to style our webpage. In addition, we were able to collaborate with VS Code’s live share to improve our efficiency.
What's next for D’éclaire
Looking forward, we plan to expand on the database for the dishes to encompass more international cuisines and also adapt our menu features depending on the country. Moreover, we plan to allow users to put in their preferences and allergies so we have D’éclaire recommend dishes to its users. Finally, since at its heart, D’éclaire is an app meant to connect people from all over the world, we hope to include more languages and culturally relevant information pertaining to food.