To give musical artists and their fans a new interactive way to connect and co-create on a spatialized platform
What it does
SAAXR is a mixed-reality experience that allows users to visually play and remix music. Take the Bass jam from one song and mix it with the synth from another. SAAXR is full of musical items from musicians all around the world. You can play and loop them in an AR environment. The remixed experience can then be shared live through the webApp in Magic Leap's 'Helio' Browser, for a visually, interactive and magical concert. The idea behind this application is to invite artists to create an AR immersive experience from their song, then others can take elements from other artist’s work and combine them to create new music and interactive AR art from the component parts. Come to play and remix in AR with SAAXR.
How we built it
We built the interaction system, music management, and visualizers in Unity. It was written in C#. We created a dynamic and interactive experience to demonstrate how artists could use the platform to remix and play in AR. SAAXR was built in Unity, Lumin SDK and webXR Magic-Leap prismatic.
Challenges we ran into
In order to prove the concept, we needed to create a variety of animated elements to sign to every sound. Those animated components also needed to co-exist in the same environment. Something that is easy for sound but not for visuals (sound can be played on top of one another, visual can't). Therefore, we were looking for quick ways to animate elements and spread them in the scene in a comfortable way to play with them in AR. We used tools like Maximo for quick animation, downloaded pre-existing 3D models and used sound visualizers to visualize sound in a quick nature. We learned that in order to make SAAXR into an online, exciting remix tool in AR, we need to offer our users a library of quick solutions to visualize their sound.
Accomplishments that we're proud of
We are proud of our teamwork and time management. Every member did his part in harmony. Geri took care of the music while Donald did the Unity scripting. Nir and Dolce worked on the animated assets as we brought her vision to life. This was a very pleasant and fun developing experience.
What we learned
As a team we learned that it's relatively simple to develop for the Magic-Leap and that the vision to create an online AR musical social app is possible. The online tools from Magic-Leap are enabling a good workflow and with fast internet, users could mix and play together in AR. As individuals, each got to learn and experiment with new skills. This is the first time for most of us are developing for the Magic-Lead.
What's next for SAAXR
After working out the prototype, SAAXR now can grow into an online, AR social platform. We need to build a wide library of 3D models and sound visualizers so users cloud visualize their sound. Then, enabled by the 5G internet, we will need to build a server that can host users from all around the world to come together and make music in AR.