Taco Bowl

We call it taco bowl, because by making lovable for tiktok we wanted to name it tok-able, and that sounds like taco bowl

Inspiration

We were ideating, and one of us mentioned how inefficient it felt to make a certain style of tiktoks. If you've ever seen a tiktok where some person makes a face, and it cuts to a clip of some meme video that has the exact same posing and energy, you probably didn't realize it can take up to days just to find the right meme clips to match your idea. It's an unsolved problem that has many begging for a solution 1

What it does

We cut down the time it takes to find the right clips to build your video from days of searching to minutes of posing. Instead of just searching with natural language, you use body language. You make a pose in front of our app, and it'll show you the clip you're thinking of but can't name.

How we built it

We used media pipe to construct the landmark representations of human body language, and perform a cosine similarity with the vectors of the representative keyframes of our library of meme footage to find the top N similar footage.

We take these results generated in our flask backend and present it on our vite frontend

Challenges we ran into

Like most other submissions will probably mention, we had wifi issues. We managed to barely escape building nothing this hackathon by spending our offline hours planning, and our online hours ferociously coding. We also designed our project to circumvent network issues.

We also ran into issues of seemingly different clips being classified into similar poses, we fixed this by performing cosine similarity across several keyframes, and establishing landmarks in more defining regions (like eyes, and hands) to better differentiate similar clips.

Accomplishments that we're proud of

Neither of us had any experience with media pipe, nor had we coded without AI in a while, but to have written >200 lines of code without AI or documentation (due to no wifi) we are proud.