Inspiration
In an old conversation, I once heard that Mixed Reality is like Graffiti ; you put your tag on the world. I thought that this was a wonderful thought. So I wanted take the moments we sing to ourselves, and turn this into a more playful experience through the Quest and help us leave our small marks, and turn that into a space that we created. And what better thing to make than flowers?
What it does
Our app takes the recording from the quest and transforms that sound into an instrument of your choice. Then you can take the 'seed' created from this recording and throw it in the room. When the seed hits a room collider it grows into a flower based on the instrument. The sound is spatial, adding to the spatialized experience.
The result is a small garden built with your voice giving soul to the flowers. And a small concert in your room for yourself.
How we built it
I had experimented with the kits.ai API before, and thought this would be a good jumping off point. The hand tracking, room mesh, passthrough and room occlusion were all gotten from the Sample Scenes.
The overall flow was more or less copied over from traditional loop pedal apps, as we thought this would be the easiest way of creating loopable music.
The 3d models and animations were all made in blender. VFX were mostly from the Unity Asset store.
Challenges we ran into
Making the API work took a lot of the first day. Some settings were wrong with Unity in the beginning, which gave us some headache since it was difficult to know why certain things weren't working.
We still didn't figure out how to properly do concurrent hands/controllers, as it feels like when it loses track of the controller, the hand on that side stops functioning properly.
Accomplishments that we're proud of
The shaders for MR was challenging, post processing isn't available in MR. So getting the objects to look 3d and not flat was a huge challenge.
Other than that, we're proud that we made fun in XR. As we strongly believe that a driving principle of XR should be that the content should be engaging in a way reality cannot.
What we learned
I learned a bit more about the updated MRKU tools and how it's much easier to find things such as walls and ceiling, as well as bounds of a room. It makes interacting with reality a lot easier.
We learned how much setting a core activity for use in MR and keeping to that basic flow makes everything easier to talk about and design.
What's next for Sound Garden
This is said for a lot of games, but having multiplayer with anchors would make this much more compelling, as it would allow us to play with someone else in our space.
Adding more instrument types would also be fun, but would require us to train our own models.
Adding more garden 'effects' would make the space much more fun.
Log in or sign up for Devpost to join the conversation.