Cheerio

• 

  Icon

Inspiration

Throughout the world people's attention spans are getting shorter and shorter, because of insane editing skills, or because of the constant rush of information. Our app solves that while still leaving some of the fun parts of short term entertainment, such as having a scroll view, but it is horizontal, without the negative effect of lowering attention span.

What it does

Our app works buy getting a youtube sport tournament (wether it be tennis, cross-country, etc) and uses ai to find the highlights and give the highlight file back to the user, where the user can rather view it or download it and watch it offline (as it is locally downloaded) or the user can also use our horizontal recreation of the obsessive scrolling we have grown so accustomed too, filled with all their downloaded videos.

How we built it

We used python as a backend (frameworks: yt-dlp, fastapi, tensor low, liberals), we used yt-dlp in order to download the videos in order to cut them into the highlight, fastapi as our api service, and tensorflow + liberals to actually create the ai that identifies if it is a highlight clip or not, through sounds and .wav files. And then combines all the clips deemed worthy (underneath or max), to make one good highlight.

Challenges we ran into

We ran into a bunch of challenges, with the backend and the frontend, for example in the backend the audio was not in sync with the video, causing horrible highlights as the audio was delayed by a few seconds. Funny enough we had the same problem with front end where the audio was not in sync with the video. Another backend struggle we went through is figuring out how to get the highlight clips, as we needed to come up with a way to see if the sound loud enough to be considered a cheer, but also identifying what is a cheer, since other loud sounds would confuse the model, and for the frontend we were given the video as a link and we had to download it, that took a while to figure out as it was sending the right link in the backend but in the frontend we were getting something completely different.

Accomplishments that we're proud of

We are proud of the model and how well it is able to process a video and return a highlight to our frontend, as well as we are proud of the rendering and how well it worked, as well as how efficient we got with storage.

What we learned

We learned all about the coding language we had chosen, because of how far we had to push the languages for the project to actually work well, we learned all about optimization of storage, cnn models for identification, and how to make decent ui wishing a short period of time.

What's next for Cheerio

We believe what's next for Cheerio, is for us to expand and share our message, improve the accuracy of the models, improve all our ui and animation to near perfection, as well as incorporating webrtc to live stream the video to the ios device. Complete support for visionOS and watchOS, as we were only able to support macOS, IOS, IPadOS during the time period given.

Built With

• audiotoolbox
• avfoundation
• avkit
• combine
• corehaptics
• fastapi
• foundationdb
• librosa
• python
• swift
• swiftui
• tensor-flow
• webkit
• yt-dlp