HarmonyShpere

Product Logo
Playing Drum
Playing Guitar
Playing Bass
Playing with Cute Characters
Place sounds anywhere you want!

Inspiration

HarmonySphere is an MR music composition tool, designed with the future of MR music experience in mind. Instead of composing stereo music solely with desktop compose software, it introduces visual spatial sounds and virtual instruments to make the music composition process more enjoyable.

What it does

In HarmonySphere, you can play instruments like the guitar, bass, drums, and more in the MR space. The sounds you create are recorded as sound loops. By placing each instrument anywhere you like, you can fill your space with your music. Additionally, cute characters will dance along to your music.

How we built it

HarmonySphere was created in Unity 2022, using various Presence Platform features.

Tools

Unity 2022.3.35f
Presence Platform
- Pass-through
- Scene API
- Hand tracking API
- Interaction SDK
- AudioSDK
Building Blocks
Depth API
- Depth occlusion
Blender

Challenges we ran into

First, we focused on creating the instruments. This initial step was a significant hurdle: we found it challenging to synchronize sound sources, record and play them back, and manipulate them within the MR space. We overcame these obstacles by drawing inspiration from sound games that felt asynchronous yet synchronous. Next, we encountered a sound issue specific to Unity and Android. Meta Quest struggles to produce multiple sounds simultaneously, and a straightforward implementation often resulted in no sound. However, by referring to sound games, we resolved these problems by using an object pool of audio sources to control the sound. Finally, we encountered an issue with the overall operability of the application. While we achieved a good design for the instruments, there are still many points to discuss regarding the application's overall operation. What is implemented now, though not perfect, represents the best solution we found in a limited amount of time.

Accomplishments that we're proud of

We take pride in the natural and intuitive design of our musical instruments, especially the guitar and piano. With the use of spatial anchors on a desk and Poke Interaction, you can experience realistic physical feedback when pressing the piano keys. You can also play the guitar with multiple string strokes, making it easy to create guitar accompaniments. Additionally, our team mascot will be there to help you enjoy composing!

What we learned

We look forward to the future of experiencing music in the MR space. First, you can fully experience playing instruments even hands-free. Second, creating music and filling the entire environment with sound is a revolutionary way to utilize space. Most importantly, visual effects synchronized with the music make incredibly enjoyable! We are convinced that the music genre is captivating, even for a small-scale idea like this one, and that this content is very much a part of the future of MR.

In this hackathon, we aimed to create the MVP for the music composition experience in the MR space. In the future, HarmonySphere will evolve into not only a composition app but also a communication app in MR.

Specifically, we envision the following developments: