Theremin

Inspiration

• insomnia, urge to play but not wanting to wake roommates and neighbors
• the one instrument that is obviously improved by VR, not suffering from audio latency
• experience with real Theremin and its challenges
• interest in tuning systems and microtonality/xen

What it does

It's as close to a real theremin as you can get in mixed reality, with some game changing features that can only be realized in MR! It shows a hologram of where the notes are, so it's much more beginner friendly. Even a beginner can make cool noises and play simple melodies right from the start.

How we built it

Years of iterating on music software in Unity. It came from a PC app which remote controlled a Max/MSP patch and custom spatial VST controls in REAPER DAW. I had to learn FAUST to port it to the Quest.

Challenges we ran into

VR dev is hard!! Getting it to feel nice and polished took a long time!

Accomplishments that we're proud of

I actually use it regularly! I missed my train to NYC the other day, and had to wait an hour. Luckily I had my headset and the new looper function added, so that hour flew by while I had a fun and engaging little improv jam sesh.

What we learned

It's possible to make a real instrument on the Quest! (of course it has been, as I'm a fan of apps like Virtuoso). I learned how to write a synthesizer from scratch in Faust and import it into Unity. I learned a lot about how theremins work and how they synthesize sound, and aimed to be true to life in my synthesizer design.

What's next for Theremin

Learning modules, holograms for alternate tuning systems, remote multiplayer, customize the look and feel, presets, etc.

Competition Updates

For the competition, I've added a couple innovative and powerful tools to aid in live composition, performance and content creation:

A mixed reality mirror overlay shows off that you are really performing live. Mixed reality without a green screen! Stand in front of a real mirror in your room and see yourself in passthrough. Position and crop the MR overlay mirror in the same place as your real mirror. Now you also see the virtual Theremin in the mirror as if it's really in the room with you. Use the Horizon OS camera to record a selfie video then share your performance on social media.

A looper function that lets you be your own orchestra. Set the tempo and turn on the metronome. Your controller will pulse with each beat. Tap A, and the loop starts recording on the next beat. Play your part, then tap A again, just before getting back to the first beat. A new Ghost Theremin will appear in front of you and seamlessly begin repeating your performance while you may continue to play along with your past self. This is so much fun, and a huge boon for creative improvisation. It's not a simple audio recording, but rather your original motion is captured and re-performed with live synthesized audio.

Built With

• faust
• metaaudiosdk
• mrtk3
• unity