Aurora Zen

Using hand tracking to paint the sky
Relax among the mountains
Change the colours through an ingame UI

Inspiration

A close friend was going through a difficult period and was using different types of mindfulness apps on their phone. I was inspired by the mechanics of some of those apps, particularly ones which used fluid and water interaction, but was underwhelmed by their visuals; I felt I could make something more visually compelling than the apps they were using and wanted to make that for my friend. I was inspired by Aurora's and how interesting it would be to draw well-rendered Aurora's in the sky. I first developed the app for Mobile a few years back. When I heard about this competition I thought it was a great opportunity to finish the project and fully port it to VR.

What it does

Aurora Zen lets you paint Aurora's in the sky using either controller support or hand tracking. Spread your fingers and drag your hand across the sky and watch it light up as you’re immersed in different peaceful environments such as snowy mountain tops or tranquil deserts. There are also controls which allow you to change the colour of the Aurora in each scene.

How we built it

I built it in Unity first for mobile using touch screen inputs. I was curious how interesting it would be in VR, and so I quickly swapped in the VR camera rig in Unity. It did require substantial updates, porting to URP and re-implementing UI and input handling.

Challenges we ran into

It was challenging to get the correct direction & location to spawn the ribbons in the sky using the hand tracking rig. I tried different vectors such as index finger, average wrist direction etc… I eventually settled on spreading your fingers and using the direction supplied by the UI selector (which is between the thumb and index finger). This proved the most natural and intuitive. Further to this, getting the sky smoke/fluid simulation to run efficiently on all types of hardware was a challenge (i.e. the blue "smoke" you see in front of the Aurora ribbon). I tried different versions of them such as compute shader versions of 3D textures, stacked textures, etc... In the end I decided a simple plane was sufficient and used a special pixel shader version of the fluid simulation.

Accomplishments that we're proud of

I'm very proud of not only the visuals but also of getting handtracking feeling as good as it does. It really grounds and connects the player to the experience.

What we learned

I learned a lot about handling hand tracking data and the flexibility, resilience and limitations of that system. I was amazed how quickly I was able to get Hand tracking working on the Meta Quest. The tools provided by Meta and Unity are very mature and allowed me to very quickly take a mobile app and bring it into VR with alternative input mechanisms (like hand tracking) very easily.