MiXR - Spatial Synthesizer

• 

  MiXR Studios logo.

• 

  Another view.

• 

  Objects in the space.

• 

  Photo of the team who created this!

*** Put on headphones for the video! It is a spatial audio experience! ***

Abstract

With MiXR, we set out to use VR technology to create an acousmatic experience unlike anything the real world has to offer. Only through exploration and play do MiXR's underlying systems emerge. MiXR is the ultimate audio-visual synthesizer that maps the unique ways in which individuals move through space to a generative environment.

With MiXR Studios, everyone is an artist.

Design Inspiration

We set out to allow people to experience music in an abstract way rather than using 3D representations of real instrument shapes (like floating disks to look like a drum pad, or a set of rectangular keys for a keyboard). Some of our favorite games, such as Beat Saber, allow you to enjoy music through movement. We wanted movement to be at the core of how you explore the sound.

For the visuals, we wanted to create a world that visually represented the electronica sound of the synthesizer through a vaporwave 80s gamer theme. You start the game within a gamer space where you can interact with the objects in the room, creating sound with them in ways you wouldn't expect, effectively orchestrating a blend between the "real" and abstract worlds.

What it does

Users are placed in a neon-lit studio and are encouraged to interact with objects in the scene. Movement and interaction with each object creates a unique sound that is meant to invoke the feel of the object.

How We built it

We chose Unreal Engine for the graphics fidelity and the ease of using Blueprints to create logic. Blender was used to create some of the 3D models while some others were either default StarterContent assets from Epic Games or free from CGTrader. Lighting, room layout, and overall design of the space was original. Some audio assets were created in Ableton Live. The full scene composition, however, was built from scratch.

MiXR builds on the new audio engine introduced in Unreal Engine 4.19 which supports modular and granular synthesis. With full body tracking, we are able to map the entire user's body to parameters of the synthesizer, as well as control shader effects to bring the audio to life.

Challenges We ran into

Most of us were using Unreal Engine for the first time ever! So we stumbled a bit at first getting things to work. But with the amazing guidance of mentors and a bit of persistence, we made it work!

Accomplishments that We're proud of

Learning Unreal, custom neon glowing art (without the performance issues!), real-time audio interaction, hand-designed feet, algorithmic chord composition, generative audio, and surviving Git merges.

What We learned

Most of us were just learning Unreal for the first time, but in particular Blueprints, scene rendering, prepping assets for import, creating emissive materials instead of adding more light to improve VR FPS, editing on sublevels, etc.

What's next for Mixr

Different environments to match new soundscapes (e.g., greenhouse for meditative, stage for rock, etc.), even more object interactions to modulate sound, and a multiplayer experience.

Built With

• blender
• steamvr
• unreal-engine