Inspiration

Traditional digital audio workstations (DAWs) force producers, sound designers, and mixing engineers to arrange complex three-dimensional audio scenes on flat, two-dimensional screens with a mouse. When mastering audio for immersive films, virtual reality experiences, spatial podcasts, and Dolby Atmos music, this creates an unnatural cognitive friction. We built StudioSphere VR to transform the producer's physical room into an infinite, holographic mixing studio using Meta Quest 3's passthrough and high-precision optical hand tracking.

What it does

StudioSphere VR is a spatial audio production canvas and tactile DAW controller built natively for Meta Quest devices.

  • Direct 3D Spatial Audio Positioning: Physically grab individual audio stems (vocals, drums, synths, effects) in mid-air and position them in 360-degree space with real-time HRTF binaural acoustic rendering.
  • Hands-First Control Surface: Adjust parametric EQ bands, compression thresholds, room reverb sends, and gain levels using natural finger pinches, rotaries, and spatial sliders without physical controllers.
  • Bidirectional DAW Synchronization: Connects seamlessly over local Wi-Fi to Logic Pro, Ableton Live, Pro Tools, and Reaper via high-speed OSC (Open Sound Control) and MIDI 2.0.
  • Mixed Reality Passthrough: Allows creators to retain complete visual awareness of their physical studio monitors, keyboards, and analog gear while spatial sound nodes float around them.

How we built it

  • Meta XR Core SDK: High-fidelity optical hand tracking using native finger skeleton gestures (pinches, grabs, palm menus).
  • Meta Spatial Audio SDK: Real-time acoustic binauralization, room reflection modeling, and occlusion physics.
  • WebXR & Three.js / Native Runtime: High-throughput spatial visualizer rendering 60+ audio stem nodes at a fluid 90 FPS.
  • Zero-Latency Network Protocol: Custom UDP socket bridge transmitting sub-10ms parameter synchronization between Quest 3 and desktop DAW hosts.

Challenges we ran into

Achieving sub-10 millisecond end-to-end synchronization between Quest 3 hand gestures and desktop audio buffers required writing a highly optimized UDP telemetry layer to prevent buffer under-runs and audio phase distortion during fast physical hand movements.

Accomplishments that we are proud of

  • Delivering a truly hands-first professional creative workflow where sound designers can intuitively shape an entire surround sound mix without holding controllers.
  • Seamless multi-stem spatial positioning with accurate acoustic physics rendered directly on Meta Quest 3 hardware.
  • Native integration with existing professional music production software pipelines.

What we learned

Hands-first spatial computing unlocks an unprecedented degree of tactile intuition for creative audio work. Being able to "touch" and reposition sound in physical space drastically speeds up spatial mixing workflows compared to traditional multi-panel mouse automation.

What is next for StudioSphere VR

  • Launching on the Meta Horizon Store in Q1 2027.
  • Multi-user collaborative mixing rooms allowing audio engineers and artists across different continents to master audio together in real time.
  • Direct spatial stems recording from hardware synthesizers and analog microphones.

Built With

  • daw
  • hand-tracking
  • horizon-os
  • meta-quest-3
  • midi.js
  • mixed-reality
  • music-production
  • osc
  • spatial-audio
  • three.js
  • webxr
Share this project:

Updates

Submission history