Celebsays

Splash Screen
Prompt UI & Video Generator
User Flow

Inspiration

Have you seen videos like "Donald Trump singing Havana" getting 100 MILLION+ views? Or want to see a celebrity saying something specific? Usually, you would need dozens of hours to find these videos, listen to them to find the words you want, trim them, merge them, etc. But with Celebsays, you can do it with a few words and a click!

What it does

Celebsays allows you to create a merged clip of your favorite celebrities based on the prompts you enter. Need a helping hand on coming up with a funny prompt? Our autofill feature powered by Co:here can help you with that!

How we built it

First, we used the yt_dlp library to download youtube videos of celebrities as mp4 and mp3 files given a link. Using AssemblyAI's transcript tools to get the transcript from the YouTube videos into a json file, AND we'll separate the speakers in a video if there are multiple, so we can ensure the clip we're taking for the merge video will just be the celebrity. Next, we used cohere API prompt generator for extra funny prompts. Afterward we'll match the prompt words into the transcript and extract the seconds where the celebrity says the phrase. Afterward, we trim the video to that moment. Do this for every word in the prompt, merge it, and you have your final video!

Challenges we ran into

Figuring out how to programmatically download youtube videos and audio files, grabbing the transcript, cleaning the transcript, and then cutting and merging the video clips based on the words together was a daunting and very ambitious task. To add to the time crunch, we came up with this idea exactly at 4pm on Saturday... Plus, this was Jason's first hackathon (he had lots of fun and came very clutch!) and Jason and Josh's first time using Flask and Python to create a web app, but with Cyra's guidance and expertise the team experience was much smoother. Cyra was also very jetlagged from her trip from halfway across the world, but she showed resilience and pulled through to create a wonderful front-end and work on Flask. Clara our designer had the challenge of figuring out how to represent our project visually through design, but more importantly validating the user case and story, and coming up with a user flow.

Accomplishments that we're proud of

We thought that completing this project in less than 16 hours would be impossible. We were kind of right, but mostly wrong. We're surprised at how much we were able to accomplish despite the late start, and am very proud of the fact that we got video download, transcription, cutting, merging, and a whole front end integrated with backend set up.

What we learned

We learned a lot about web development using Flask and Python, and especially about working with APIs. The AssemblyAPI was very accessible for us, and came with great tutorials we used to figure out how to transcribe youtube videos.

What's next for Celebsays

We plan to refine our product by providing better clip padding (spacing between words), full implementation of speaker diarization with AssemblyAI's API combined with Voice Identification to better identify speakers.