Get In My Belly

The landing page
Ingredients and quantities summarized neatly
Instructions summarized with screenshots from the video
Example of our Trie structure for parsing ingredients
Tries for ingredient parsing

Inspiration

2020 had us indoors more than we'd like to admit and we turned to YouTube cooking videos for solace. From Adam Ragusea to Binging with Babish, these personalities inspired some of us to start learning to cook. The problem with following along with these videos is that you have to keep pausing the video while you cook. Or even worse, you have to watch the entire video and write down the steps if they're not provided in the video description. We wanted an easier way to summarize cooking videos into clear steps.

What it does

Get In My Belly summarizes Youtube cooking videos into text recipes. You simply give it a YouTube link and the web app generates a list of ingredients and a series of steps for making the dish (with pictures), just like a recipe in a cook book. No more wondering where they made the lamb sauce. :eyes:

How we built it

We used React for front-end and Flask for back-end. We used Youtube-Transcript-API to convert Youtube videos to transcripts. The transcripts are filtered and parsed into the resulting recipe using Python with the help of the Natural Language Toolkit library and various text-based, cooking-related datasets that we made by scraping our favourite cooking videos. Further data cleaning and processing was done to ensure the output included quantities and measurements alongside the ingredients. Finally, OpenCV was used to extract screenshots based on time-stamps.

Challenges we ran into

Determining the intent of a sentence is pretty difficult, especially when someone like Binging with Babish says things that range from very simple (add one cup of water) to semantically-complex (to our sauce we add flour; throw in two heaping tablespoons, then add salt to taste). We converted each line of the transcription into a Trie structure to separate out the ingredients, cooking verbs, and measurements.

Accomplishments that we're proud of

We really like the simplicity of our web app and how clean it looks. We wanted users to be able to use our system without any instruction and we're proud of achieving this.

What we learned

This was the first hackathon for two of our three members. We had to quickly learn how to budget our time since it's a 24-hour event. Perhaps most importantly, we gained experience in deciding when a feature was too ambitious to achieve within time constraints. For other members, it was their first exposure to web-dev and learning about Flask and React was mind boggling.

What's next for Get In My Belly

Future changes to GIMB include a more robust parsing system and refactoring the UI to make it cleaner. We would also like to support other languages and integrate the project with other APIs to get more information about what you're cooking.