Rolling Notes

Gameplay Visualization
Player Tapping Buttons on Embedded System

Inspiration

Our team members each have a music background and enjoy rhythm games, so we aimed to create a project that combined our knowledge in hardware and software with our passion for music and thus Rolling Notes was born. Fun fact: Rolling Notes comes from Rolling Stone, since Notes is an anagram of Stone!

What it does

Rolling Notes is an immersive game, engaging both hands in its rhythm challenges. When a button is pressed, the corresponding color flashes on the LEDs, illuminating the game experience. The game detects a player's inputs and categorizes them as perfects, goods, hits, and misses. Each of these markers contributes to the score multiplier and encourages players to compete for best accuracy. Along with the traditional beat taps, our game has a revolutionary glissando addition which slides across the different notes. For these notes, the player must engage their other hand to work the potentiometer and follow the path to maintain their score multiplier. At the end of the game, the player's accuracy is revealed and they are given a rank to flex to their peers.

How we built it

Beginning by interfacing the Arduino ESP32 to a breadboard, we connected the Arduino to an LED strip, an array of buttons, a potentiometer, and an LCD. Then, by use of SPI protocols, we were able to facilitate communication of data between the Arduino and Unity. The rest of the game aspects were built in C# via Unity, with the game's graphics designed with Krita. The song chosen for the game's note map was APT as it was a lively and familiar tune that players could vibe along with.

Challenges we ran into

We ran into three major challenges, the first being accurate collision detection. We were able to solve this problem by using a variety of hit boxes. We also had trouble with processing data sent from the Arduino. To tackle this issue, we implemented a new method of sending data on the buttons, potentiometer, and reset in a single string separated by colons. By splicing the string, we were able to successfully transfer this data between other components of the project. Finally, we needed an effective way to translate potentiometer data to in-game movement, which we accomplished by use of factor manipulation.

Accomplishments that we're proud of

The LED strip offers an additional immersive quality that we feel is unique to our game as it gives the player a sense of being surrounded by the game environment without a virtual headset. Additionally, our SPI's ability to simultaneously read and communicate data between Unity and the Arduino and vice versa is something we are incredibly proud of.

What we learned

We learned how to utilize the different Unity namespaces and packages in game development. This includes the use of gameObjects, scenes, resetManager, and System.IO.Ports.

What's next for Rolling Notes

We hope to expand the song library as well as increase the scale of the project to make it comparable to beloved games like Dance Dance Revolution. Furthermore, we hope to add varying difficulties for the songs so that our game is friendly to newbies and pros alike. In the future, using Generative AI to create beat maps for different songs would allow us to expand our library at a much quicker rate.