As twitch viewers ourselves, we noticed that the most entertaining streams to ones were those in which the streamer interacts with the viewers. Notably, one of the biggest spikes in attention that we noticed were when the viewers got to dictate a challenge to the streamer. Going off of this trend, we wanted to build an extension that allows the community to interact even more with the streamer on this deeper level of gaming prowess. With some more thought, we came up with as many creative ways to alter the gameplay of the streamer as possible.
What it does
We created a panel extension that comes with a user-friendly startup guide for the streamer, and clear instructions on the interactive buttons to the viewers about what they can do to interact with the streamer. On the streamer's end, the extension works as an overlay on their screen for whatever they are doing, so this extension can work even when the streamer is not gaming! Currently, there are three modes: "Color Explosion" will cause colored circles to pop up randomly around the screen. Then "The Worst Rave" brighten it up even more as it changes the color of the screen. Finally, there is "Send a message, make it rain" where the user can type in a message, and that message will rain across the streamer's screen continuously. There is a lot of flexibility with configurations, so the streamer can manage how much the intensity/frequency of the modes and a colour-blind option. This will create an entertaining and inclusive experience for the community as they watch the streamer successfully game while being distracted, or it can be used as a way to sabotage their favorite streamer.
How we built it
At first, we tried using C++ and Python to interface with the streamer's screen. In the end, we built a frontend with electron.js which overlayed onto the computer screen, which enables the visuals to work on the screen. Then, we built a backend to handle the extension and bits donations to trigger the electron application to work on the streamer's end for the gameplay modes to activate. We used sockets in order to only allow the button to work and users to donate when the streamer activates the extension on their end, and express.js to handle routes that the frontend calls when the panel is interacted with. The interactive panel itself was built with a React framework. We read the twitch documentation thoroughly to understand how to create twitch extensions and utilize the developer rig.
Challenges we ran into
Initially we were not sure how to create an extension that can change the gameplay experience for the streamer, since the extension would have to interact on the streamer's PC. Some ideas included using OpenGL and building a C++ driver. We then overcame this hurdle by using electron.js and created a windowless application that would work as an overlay on a streamer's PC as they enable the extension to use for the viewers. We built in security features so that the extension would not work if the streamer did not activate the backend. We initially were confused about how to do this, and after a while we were able to solve this with the use of sockets.
Accomplishments that we're proud of
We are super proud that it all works! The backend and frontend are connected, we learned about sockets, and we learned about working around cross origin issues with sockets. We are also really proud about thinking about the large scale issues that could result from a twitch extension, such as many viewers connected to one streamer, if the streamer did not activate the extension, etc, so we were able to create a database and kept track of when the streamer was connected or not.
What we learned
We learned more from streamers how they like to interact with their viewers and we also able to learn many different technologies. We got familiar with twitch extensions, understanding the documentation, and how to create extensions, so that in the future, we can continue to support our extension and create new ones. We learned some technologies such as express for the backend which will handle the gameplay mode activating for the streamer combined with electron to deliver the thrilling and gameplay confusing visuals. We got more comfortable with using version control and used git to handle merge issues.
What's next for Exstream Mode
Currently, the idea is to just monetize Exstream Mode so that whenever someone donates a certain amount of bits, a certain gameplay mode will be activated for the streamer for a set amount of time by the streamer. However, at the foundation of our idea, we wanted a better way for the community to interact with the streamer, and so we believe that this extension can open the door to many possibilities, from more gameplay modes with mouse aim inversion to a party mode in the chat where users can all compete in a quiz about the streamer to get the chance to activate a gaming mode that they want to. We believe the opportunities are endless to make the stream much more interactive for viewers and more entertaining!