Inspiration

Atmospheric Audio Generator (AAG) is a VR musical instrument, sampler and multitrack recorder inspired by artists including Jean-Michel Jarre's laser harp audio experimentation and guitarist Jimmy Page's use of the theremin in live performance. Also inspired by film soundtracks such as Brian Eno's Prophecy Theme used in the film Dune (1984), Vangelis' Blade Runner score (1982) and Mark Isham's work on The Hitcher (1986).

AAGVR is based upon the 2D version of AAG I released in late September - intended for musicians, filmmakers and other people interested in experimenting with atmospheric, ambient and unusual sound.

What it does

The user controls sound using movement via an interactive light orb. The audio wave is then shaped by a series effects processors eg delay, reverb and overdrive. The goal is to enable the creation of atmospheric, ambient audio assisted by the use of hand movement without the need for musical training.

Key features:

  • Interactive sound generation using hands or Quest 3 controllers
  • Tone, filter, pitch and delay time can be controlled by hand tracking or controller movement
  • Other effects include reverb, overdrive, tremolo and harmony effects
  • RANDOMISE button complimenting hand movement with automated changes to effects values
  • An Audio sampler uses the headset microphone to sample sound
  • Two-track multitracker with track trimming, fades, volume, mute and solo
  • WAV mix and stem export with high-quality audio export
  • Interactive tour and optional tooltips to help users understand functionality

How I built it

Summary: The original AAG design was developed between May and September using Adobe Creative Cloud (Photoshop, Ilustrator, InDesign). My Meta Start Developer aim was to convert it to a 3D space using the same graphic design > AI coding > testing > iteration process within the webXR framework.

Detail: The design and overall identity of the original AAG was established. The 3D VR version needed to retain the look and feel of the 2D version but take advantage of hand tracking and other Quest 3 headset features available eg microphone. Some functionality, such as the reduction of 3 recording tracks on the multitracker were necessary to prevent overloading.

A conversion was planned by creating a precise specification with assistance from ChatGPT (analysis of the 2D site). Once the basic conversion was complete, there were many elements that required refinement, rethink and additional features. This was followed by an intensive phase of testing and improvement to both the audio and visual aspects of the project.

Challenges

  • Some code took several attempts to get the intended functionality despite clear images and instructions

  • Audio playback whilst switching between VR and desktop versions within the headset caused audio glitches. Some operations can introduce short interruptions or processing spikes. This is an ongoing challenge.

  • Balancing functionality with the limitations of the headset: some features needed to be reduced eg number of recording tracks and shadows cast from the light source to improve text clarity

Accomplishments

  • The interactive orb: rather than simply recreating the original 2D interface on a panel, 3D movement using hand / controller tracking controls 4 wave shaping features

  • Sound sampling from the Quest headset mic allows the user to play the instrument using any sound they can record on the headset. Built in trim + fade tools provide additional control.

  • Layering sounds and exporting the results as .wav files allows potential professional use in soundtrack production.

What I learned

  1. I was delighed to discover that the headset could control and record so many audio / visual parameters whilst using hand tracking in a VR immersive environment. This helped me learn how far I can push the headset when designing new experiences.

  2. Refinement of the concept > design > specification > AI coding > testing > refinement process I have been developing in the past few months.

  3. Despite microphone placement that is less than ideal for recording, Quest 3 is able to record audio of a suitable clarity, frequency balance and quality to be used as a loop sample for the instrument.

What's next for AAG...

Potential future developments include:

  • obtain feedback to establish approach, useful features and difficulties
  • additional audio effects such as auto-wah, chorus and compressor
  • improvements editing functionality making it easier to grab trim and fade handles
  • adding additional hand gestures to control the audio
  • porting to a professional version as a standalone application in the app store

Built With

Share this project:

Updates

Submission history