Inspiration
My Inspiration is Intersection of Art and Technology
The concept began when I came across a review of high-end open-back headphones that explored their ability to create an immersive listening experience. Years later I became fascinated by the visual artistry, stage design, and audiovisual experiences of music festivals.
These experiences led me to a deeper realisation: music is not shaped by sound alone, but also by the environment in which we experience it.
What it does
OrcaTrack Transforms music into an immersive experience, Combines immersive 360° visuals, virtual environments, and six-directional spatial audio, above, below, left, right, front, and behind, to create a fully immersive musical experience.
As users turn or move their heads, the sounds they hear change based on their orientation and position, allowing them to experience music from different directions and feel completely surrounded by a 360-surround system.
The ultimate goal is to transform music listening from a passive activity into an interactive experience.
How we built it
Meta Spatial SDK (Programming): Used to develop the VR application, implement interactive features, integrate 360° visuals and spatial audio, and build the final APK for Meta Quest.
Ableton Live (Music Production): Used to compose, produce, edit, and mix music and audio tracks for the immersive sound experience.
Final Cut Pro (Visual Editing): Used to edit and produce the 360° video content that forms the virtual environments.
FFmpeg (Video Processing): Used to convert, encode, and optimise video files for compatibility and playback performance.
Challenges we ran into
- Implementing 3D directional spatial audio
Implementing 3D directional spatial audio was more challenging than I initially anticipated. I experimented with several approaches but struggled to keep multiple audio tracks synchronised, as they would gradually drift out of sync. I eventually solved this by ensuring all tracks had the same BPM and duration, and started playing simultaneously.
- Implementing hand tracking
Implementing hand tracking was initially one of the most challenging aspects of development, as the early implementations were extremely buggy and unreliable. I experimented with multiple approaches to improve gesture recognition. Implementing joint finger indexing solved majority of the problems faced.
Accomplishments that we're proud of
- UI Interface is very responsive
- Hand tracking behave as intended
What we learned
This is my first MetaSDK development experience, for few years I developed only on WebXR.
What's next for Alex
Moving forward, I want to expand OrcaTracks by incorporating stereoscopic 3D 360° video playback to create a greater sense of depth, realism, and immersion within the virtual environments.
I also plan to develop more immersive soundscape works.