Jazz Frog

Inspiration

Our inspiration comes from our love of frogs and groovy tunes! We wanted to make a runner game where the player tries to collect as many coins as possible, but also have the coins play a note based on an instrument of the player's choosing. We also wanted to implement an AI that determines the pitch of the notes played based on how far up and down the frog moves. In addition, we wanted to include the use of a controller to give the game a more arcade feeling to it.

What it does

The player controls a frog via controller and selects an instrument to play when the frog collects a coin. When the game starts, the aim is collect as many coins as possible. Based on how far up and down the frog moves, the pitch of the note plays changes. As you collect more coins, more song notes are played. The player has 30 seconds to collect as many coins as they can.

How we built it

The frontend portion of the project was designed using JavaScript and Phaser. We implemented coin and frog sprites: the coin sprites flow across the screen before being collected/not collected and the frog sprite stays fixed along a given x-coordinate. We also implemented collision between the frog and the coins as well as coin and frog animations. In addition, we managed to include pixel artwork that our team created and free artworks available online to give our game an arcade feel. Once the game finished, we display the final score based on the number of coins collected.

The game was programmed in JavaScript and typescript. We used a modified version of a music generation AI model from Google's Magenta Labs, which was designed for classical music so we had to make some changes for it to fit better in a jazz context. Specifically we made the model stick to a simple blues scale for now (we'd like to change this in the future) and we also found that the model would often play the same note over and over so we forced it to not play the same note twice.

Challenges we ran into

We ran into issues with the motion of the sprites and getting the sprites to be where we wanted them to be. We also ran into issues with collision and generating new coins throughout the game. In addition, we ran into issues with getting the artwork to display and animation to run properly.

Accomplishments that we're proud of

We're proud of the fact that we got a controller to work with our project, an AI which changes note pitch, and a working frontend design with artwork all during our first hackathon.

What we learned

We learned how to write a 2D arcade game in JavaScript as well as how to connect both frontend and backend portions of a project that we were initially unfamiliar with. We also learned how to implement a physical controller into the project as well as how to implement an AI to change aspects of our game (e.g., how the pitch of the notes changes as the game progresses). We're super excited that we were able to accomplish all of these during our first hackathon.

What's next for Jazz Frog

For the future, we hope to make Jazz Frog an application that players can run online and connect physical controllers to. We also hope to possibly add more game modes to Jazz Frog and allow players to play a variety of arcade games without having to worry about how their controller needs to connect to the application or installing additional extensions that many online games typically require. We also hope to work further with AI models to make jazz music more accessible to people who don't have the time to study countless hours of music theory and just want to have fun.