DrumStar

DrumStar is a VR drum game for fun or practice: play a full kit in your own virtual auditorium, headphones on, no noise complaints, with hits that feel real.

Comment

Inspiration

DrumStar started from a simple idea: what if learning and playing drums in VR felt less like a menu-driven rhythm game and more like standing in front of a real kit?

Drums are physical, loud, expressive, and hard to fit into everyday spaces. Meta Quest gives us a way to bring that feeling into a room without needing the real hardware, the noise, or the space.

We wanted DrumStar to feel immediate: grab the sticks, see the kit in passthrough, hit the drums, and hear the response. The goal is to turn mixed reality into a playable music space.

What It Does

DrumStar is a mixed reality drumming prototype for Meta Quest. It places a drum kit in front of the player using passthrough, lets the player pick up virtual drumsticks, and plays drum sounds when the stick tips strike the correct drum surfaces.

The current kit includes:

  • Snare
  • High tom
  • Mid tom
  • Floor tom
  • Hi-hat
  • Crash
  • Ride
  • Bass drum

The hi-hat supports open and closed states, and the kick drum is triggered through controller input, giving the player the basic pieces of a playable drum set.

How We Built It

We built DrumStar as a native Android/OpenXR app for Quest using C++, CMake, the Android NDK, OpenXR, OpenGL ES, and Oboe for low-latency audio.

The project runs through NativeActivity, with the gameplay, rendering, input, collision, and audio systems all handled in native code.

The drum kit meshes and textures were created with Meta Spatial Editor, then exported into the app as GLB assets. We load the models directly, render them with custom OpenGL ES shaders, and place them in a passthrough scene so the kit feels like it exists in the player’s real room.

For gameplay, we track the player’s hand and controller poses through OpenXR actions. Each drum has a position, radius, orientation, and sound ID. Each frame, DrumStar checks whether the tip of a held drumstick crosses a drum plane with enough downward motion. When it does, the audio engine triggers the matching WAV sample.

For sound, we built a small drum audio engine on top of Oboe. It loads individual WAV files for the drum pieces and mixes active samples in the real-time audio callback.

Challenges We Ran Into

The hardest part was getting the feel right. A drum game only works if the hits feel connected to the player’s hands, so we had to tune stick placement, drum positions, hit radii, collision planes, and motion thresholds.

OpenXR input also had sharp edges. One crash came from using an unsupported hand interaction path for pinch input on the Quest runtime. The app would build and launch, but fail during action binding. Fixing that meant learning which interaction profiles and input paths the runtime actually supports, then binding grab actions to supported controller squeeze/select inputs.

Audio timing was another major challenge. Drum hits need low latency, so we could not treat audio as an afterthought. We used Oboe and a simple sample-mixing callback so drum sounds can start immediately when a hit is detected.

Accomplishments That We're Proud Of

We are proud that DrumStar is not just a mockup. It has a real native Quest foundation, real OpenXR input, passthrough rendering, imported 3D drum assets, grabbable drumsticks, hit detection, and multiple drum sounds working together as one experience.

We are also proud of building the core interaction ourselves. The app detects actual stick-tip motion against drum surfaces instead of relying on a simple button press for every drum. That makes DrumStar feel more like an instrument and less like a flat rhythm-game controller.

What We Learned

We learned that mixed reality music is all about small details. A few centimeters of drum placement, a small change in stick angle, or a slightly wrong collision radius can change the whole feel of the instrument.

We also learned a lot about the Quest native stack: OpenXR action sets, passthrough layers, hand and controller poses, GLB asset loading, OpenGL ES rendering, and low-latency Android audio with Oboe.

The project pushed us to understand not just how to render something in VR, but how to make it respond in a way that feels physical.

What's Next for DrumStar

Next, we want to turn DrumStar from a playable prototype into a fuller rhythm game and practice tool. That means adding songs, timing targets, scoring, tutorials, calibration, and better feedback for accuracy and rhythm.

We also want to improve the instrument itself with better drum animations, stronger haptics, richer sound variation, more natural hand tracking support, improved kit positioning, and a polished visual identity.

The long-term goal is for DrumStar to become a mixed reality drum kit that is fun for beginners, useful for practice, and expressive enough to feel like making music.

Built With

  • .cgltf
  • .glsl-shaders.-asset-creation/source-notes:-meta-spatial-editor-for-drum-kit-assets
  • .jpg-textures
  • .wav-drum-sounds
  • anchor-api-permission
  • android-studio
  • androidxr
  • arm64-v8a
  • audioapis
  • but-the-renderer-code-path-i-see-is-opengl-es
  • c
  • c++-shared.-vr/xr/graphics-openxr:-khronos-openxr-1.1.49.-openxr-android-loader:-org.khronos.openxr:openxr-loader-for-android:1.1.49.-graphics:-opengl-es-/-gles3-shaders
  • chatgpt
  • cmake-3.31.1+.-android-platform:-sdk-34
  • codex
  • core-stack-languages:-c++17
  • gimp-for-placard
  • glsl-shader-code.-main-app-model:-android-nativeactivity-with-android-main
  • gradle/groovy
  • hand-tracking
  • meta/openxr-extensions
  • metaquestheadset
  • metaspatialeditor
  • min/target-sdk-34
  • ndk-28.2.13676358
  • no-java/kotlin-app-logic.-manifest-has-android:hascode="false".-ide/project-type:-android-studio-project.-build-system:-gradle-+-android-gradle-plugin-8.13.1
  • not-vulkan.-audio/assets-audio-engine:-google-oboe-1.10.0-for-low-latency-native-audio.-3d-model-loading:-cgltf.-image-loading:-stb-image.-asset-formats:-.glb-models
  • oboe
  • opengles
  • openxr-apis
  • pixabay-for-some-sound-effects.-cloud/database/api-cloud-services:-github-only-for-private-source-hosting:-neogrammer/drumstar.-databases:-none.-backend/web-apis:-none.-runtime-apis:-android-ndk-apis
  • render-model-feature.-note:-the-manifest-also-requires-vulkan-support
  • scene-permission
  • vulkan
  • with-opengl-es-3.2-required-in-the-manifest.-device-target:-android-xr-/-meta-quest-style-headset.-meta/oculus-features:-passthrough
  • xml
Share this project:

Updates