Jump On It

Inspiration

We wanted to build a jam sesh in XR, but after talking to musicians we heard the same complaint: learning and playing instruments in XR isn't very fun. There's no haptic feedback, the devices can get laggy, and the acoustics just aren't the same. Given all this, we wanted to figure out to make music more enjoyable in XR.

What it does

The experience is multiplayer, with any number of people able to join in. Players can see on the floor a giant floor piano in AR. As they jump around and play on it, they can discover secrets that build out an audiovisual world around them -- depending on what chords are played, new music and new visuals will appear.

How we built it

We used the Quest 3's passthrough ability to render the piano in XR. The game runs on Unity, and implements multiplayer with Photon.

Challenges we ran into

Multiplayer is really hard, especially at a hackathon where everyone else is also trying to use the Internet.

Accomplishments that we're proud of

The multiplayer works and people have a lot a lot of fun jumping around in the piano in 3D space.

What we learned

Multiplayer is really hard. But jumping on the big piano is really fun.

What's next for Jump On It

We want to implement new environments, like a forest or an ocean, and new interactions: being able to put the piano on the wall and throw things at it, or on the ceiling and jumping up to hit it, or even shadowboxing it!