AutoClip

Inspiration

Many streamers when they do something exciting or memorable, they tell the chat to "clip that chat". Instead of relying on the integrity of chat to do it, we put the power into the broadcasters hands where they are able to create a clip while live streaming.

What it does

When a broadcaster is live-streaming on twitch and they say the keyword "clip", our program is frequently listening for this keyword and once it hears the word "clip", it will automatically clip the last 30 seconds and display the clip in the corner of our video through a twitch extension.

How we built it

We divided up the tasks into four main parts: 1) Detect audio and turn it into text 2) From a string text, have the functionality to automatically create a clip 3) Then we created the clip returns a clip ID to the websocket for data transfer 4) From the data transfer, we are able to put the video clip onto twitch using a video-overlay extension

Challenges we ran into

Building plugins for OBS because in order to build a plugin for OBS, you need to build the entire OBS from the source. The speed of downloading and installing files took up a large chunk of our time.

Additionally, sending video and audio data turned out to be a larger issue than we realized.

Accomplishments that I'm proud of

We were able to get speech recognition down thanks to the Google Cloud Speech-to-Text API. We were also able to figure out how to create plugins for OBS through scripting languages such as Python.

What we learned

We learned that the doing the DevOps for OBS is a huge time commitment to get working.

What's next for ClipThat

Built With

• google-web-speech-api
• obs-studio
• python
• twitch-developer-rig
• websocket