FromTheSidelines

• 

  Landing Page

• 

  Upload Trickshot Section

Inspiration

My friends and I would make a lot of dude perfect parodies where we would try to do impressive layups, dunks, lobs, etc. Some of them turned out to be actually impressive, but the video alone didn't capture the intensity. This is different from NBA highlights because the commentary and crowd noises enchant the dunk or shot, so I brought those crucial elements to our own trickshots.

What it does

The app takes a video of a basketball trickshot, and overlays the video with crowd noises akin to an arena, andESPN style commentary that accurately describes and hypes up the moves in the video.

How I built it

I used a collection of AI APIs for implementation. I used TwelveLabs for video summarizing in order to get accurate timestamps and description of the trickshot and what exactly is happening. I then fed that summary to Gemini, where I told it to generate a script that an excited sports commentator would say. Then I fed this script to ElevenLabs which gave me a voiceover of a hyped sports commentator. I also used ElevenLabs to generate crowd noises. I then used ffmpeg to overlay these two audio files onto the original video, which was then returned to the user.

Challenges I ran into

API Integration Complexity

Coordinating three different APIs with different interfaces was one of the biggest technical hurdles. TwelveLabs, Google Gemini, and ElevenLabs each have their own response formats and request formats. I added proper error handling to make sure that failures don't waste credits or leave users hanging.

Video Processing & Audio Sync

FFmpeg is powerful but unforgiving. I had to learn the different commands and how to use them with python to filter chains to strip original audio, overlay the commentary, and mix in crowd noise without quality loss. I had to account for unpredictable durations and build FFmpeg filters that handle variable-length audio gracefully, using fade-in effects and duration controls to blend everything smoothly.

Accomplishments that I'm proud of

I was able to deliver a fully working MVP in under 24 hours by myself, with a sleek UI!

What I learned

I learned that the APIs are there for me and it's up to me to build something innovative and useful.

What's next for FromTheSidelines

Commentary in different languages, different sports, different voices.

Built With

• elevenlabs
• fastapi
• ffmpeg
• gemini
• react
• twelvelabs