Tasty videos are cool
What it does
It takes a youtube url for a cooking show video, like Rachel Ray, or Gordon Ramsay and it generates a short form video recipe similar to the popular Tasty videos.
How we built it
We use ffmpeg to extract a couple of frames of video per second and send it to our classifying service. We used transfer learning to train a neural network to recognize kitchen and food related things from images, and a set of word vectors trained on recipes to differentiate between kitchenware and food items from the image sent by the chunker. The data pipeline tells us whether the frame at that time interval is a relevant thing we want to keep and we build small chunks of the original video that contain relevant contents. More relevance is given to images with shots of food being cooked and prepared over shots of the host and extra content. We take this collection of videos and put it together into a new shorter video that captures the essence of the original.
Challenges we ran into
Slowness in training. Training data. Putting video back together
Accomplishments that we're proud of
What we learned
M A C H I N E L E A R N I N G
What's next for Tastai
Cooking for you