Inspiration

I was inspired from the feeling of performing in front of crowds as a musicians myself. Though I'm not a big touring musician, I really felt that VR is the perfect platform to both experience the feeling of actually performing on stage and also help people perform in front of others.

What it does

RealStage teleports players onto beautiful stages, so that they can perform all types of music in front of cheering crowds. It has the following functionality:

  • Being able to play on stage with Virtual Drums or Virtual Dance Hands
  • Being able to turn on Passthrough, so that you can view your own instrument while also seeing the virtual environment around you.
  • Full Hand Tracking Support
  • Ability to import your own songs and have their notes comes down toward you in the virtual world, so you can practice real drum sequences and choreography in a rhythm-game style format.
  • Ability to stream songs or tutorial videos using your own streaming apps, so that you can practice your favorite songs or follow along tutorials.

How we built it

This was built with the Meta Fork of Unreal Engine 5 and it is still in pre-release Alpha stage. It was initially built with Controller support only, but I added full Hand Tracking support during this game jam.

Challenges we ran into

There were weird bugs with using an older version of Unreal (5.4), such has the hands not rotating properly and inconsistent clicks. But I managed to fix them with a lot of manual programming. I didn't expect that hand-tracking would reduce some FPS in my game, so while using techniques like foveated rendering and dynamic resolution is helping, I think I'll still need to optimize a bit more for the final release. But this state is running really well and there's a great path ahead!

Accomplishments that we're proud of

Considering all the real-time system that are working together, I am so glad I was able to make this game run at all at playable frame rates on a Quest 3/3s. Having heavy system like a procedural rhythm-note lane system, hundreds of crowds, a somewhat realistic environment, and real-time hand tracking working all together on a standalone device is something I don't think has been done before. There's still more work to do, but I'm glad it's all in a very playable state and I know the steps forward to polish this for final release.

What we learned

I learned all about hand-tracking honestly. It was never something I anticipated to work on ever an indie game developer. This game jam gave me opportunity to expand my development horizons.

What's next for RealStage

The near term step is Final Release - which includes further optimization, more Stages, more built-in Songs, and overall Polish. In the future, I want to use this platform to add learning content so people can learn choreography and instruments in VR with virtual instructors, and then be able to perform on stage with what they learned. I also would like to collaborate with real artists/bands and record them in stereoscopic VR; then transport them into my stages, so players can perform alongside them. I plan to use any prize money for doing exactly that: renting greenscreen studios, hiring instructors, purchasing filming equipment, and any costs of collaborating with real artists/bands.

Built With

Share this project:

Updates

Submission history