An immersive concert for Meta Quest by Alex Coulombe Presents.

Inspiration

I wanted someone who had never read a score to feel what it’s like to conduct. A conductor can change a phrase with a gesture. What if the whole landscape were listening? That idea turned a quiet dock into a stage for swans, light, and weather.

What it does

Pinch your thumb and index finger to take up a baton. Give a downbeat to wake the lake. Sweep to ripple the water, strike to call lightning, and bring both hands together to gather the flock in a shaft of moonlight. Across eight scored movements, you can shape the scene, pause to listen, or bring the program back for an encore.

A short rehearsal at the dock teaches the basic gestures. During the concert, the score on the music stand introduces each new cue. The program can also play through on its own.

How I built it

I built the lake, weather, visual effects, and flock choreography procedurally in Godot. OpenXR drives the hand tracking. Beat and loudness maps connect the player’s gestures and the scenery to the score. A synthesized arrangement of Tchaikovsky’s Swan Lake theme begins with harp and oboe during tuning and returns with strings under the moon.

Challenges and lessons

A gesture needs to feel like a musical cue even when someone is using the baton for the first time. I used the water, light, and flock as feedback and put each instruction on the stand when it becomes relevant. Large, readable changes in the lake make it easier to connect a gesture to its effect while looking out at the performance.

What changed for Meta Quest

The original free-play vignette let players ripple the lake, guide swans, light stars, and make fireworks. This adaptation adds a guided rehearsal, tuning and eight scored movements, timed changes in weather and lighting, new flock formations, and beat-driven responses. The virtual baton follows your pinch. You can use bare hands from the opening menu through the encore.

What’s next

I plan to test the rehearsal with first-time players and tune the cues and seated play on Quest 3 and Quest 3S. The target is a free release in the first quarter of 2027.

Try the Quest build · Watch the trailer

Built With

  • godot
  • meta
  • meta-quest
  • openxr
Share this project:

Updates