NOT a Walk in the Park!

Squirrel game through the glasses!
Looking through the glasses at holding the gopher with hand tracking
Editing the gopher game
Testing playing the squirrell game in the editor
Editing the birds and bees scene in the editor
Testing filming our scenes with a AR glasses capture rig
The whole team working on the project
Testing scenes on the glasses

What we made

We made a series of short mini games in the Snapdragon Spaces track, which involved working with a Motorola Android phone and the Lenovo ThinkReality A3 glasses, which feature the Snapdragon XR1 chip. Our mini games followed a common theme of “It’s not a walk in the park,” in which each one has you trying to fend off or catch various animals you could find in a park. All of these minigames also involved learning how to use the Snapdragon Hand Tracking SDK.

The first minigame we made was a group of squirrels in a tree throwing acorns at you for 30 seconds, and if you could smack an acorn away with your hand and have it not hit you, you’d gain a point. The second minigame we made was about a park with Birds, Bees, and Balloons. Pick up the pan and swat at the bees and floating balloon demons. Successfully “panning” them earns you points! The third minigame we made was a Whac-A-Mole type game where you have to try and grab a gopher running into different bushes that are located all around you.

Inspiration

We wanted to leverage AR and hand tracking to encourage people to move. We had lots of different ideas and different inspirations for them such as a parkour game like Dying Light 2, rhythm game like Beat Saber, fitness game like OhShapes or gardening game like Tools Up. If given more time, we'd love to develop our games further to innovate on those inspirations and have more hand interactions. We then settled on the idea of being nature-inspired while our mechanis we're inspired by What the bat?, a comedy VR game based around a series of short mini games exploring what life would be like if you had baseball bats for hands. The gopher minigame is an AR reinterpretation of the classic game Whac-A-Mole. Instead of just hitting animals popping up from holes, the game is around you and you’re physically turning around to try and catch an animal moving between bushes.

How we made it

We made our minigames in Unity with the Snapdragon Spaces SDK to enable builds on the Lenovo ThinkReality A3 glasses with powerful interactions such as hand tracking and pushable 3D buttons. We used github for the version control. We got most of our public domain assets from poly.pizza and the Unity Asset Store. To test our project, we had to build it into a Motorola Android phone that the glasses are tethered to using Android Debug Bridge.

What we learned

We learned a lot about working with AR SDKs to integrate our existing Unity knowledge with tools that are required to work on this unique platform. Since everything was so new, we learned everything on the fly and learned to leverage everyone's individual strengths. We learned to be comfortable with things not working and constantly troubleshoot and find workarounds.

We learned an immense amount about how to rapidly learn and develop in Unity, and how to fix a lot of the edge case problems that can happen with Github collaboration. For example, we initially set up the repo without a gitignore file, and a mentor helped us make one and granularly decide all the files we wanted to include and exclude.

Challenges we faced

Every step we took was faced with constant challenges and errors causing us to lose a lot of time in set-up and testing. The environment setup took an exorbitant amount of time. Most people in the same development track as us took at least 4 hours to set up Unity with the SDK. One of our members had to spend 7 hours on the first day to figure out how to properly download an old version of Unity with its Android tools, set up ADB file transfer and debug this pipeline.

We spent the entire remaining time on the project constantly learning new things about different parts of the SDK. The documentation was incomplete and difficult to read while sample scenes didn't work together, so we often had to reverse-engineer didn't things and go through constant trial and error, especially for hand tracking and interactions.

Even with all these considerations, the development process wasn’t the easiest because the process of testing our changes in the glasses required. Even during this process, there were a lot of slow downs, as the project would only successfully load in the glasses about half the time and we had to constantly do any combination of the following: force quit the app, restart the phone, unplug the glasses from the phone and wait for them to restart, and charge the phone when the battery died.

At one point late near the end of our hacking time, we messed up something in Github severely and lost a couple hours rolling back our most recent changes and fixing it on everybody’s computers.

Despite all these challenges, it was an amazing experience for all of us. The moment whenever something clicked was always amazing. We got so much help and guidance from the Snapdragon team and other mentors. They were absolutely amazing. And going through the same troubles as others in the track it together fostered a great connection where we'd celebrate together every time something worked--if one team cheered, others would join in on the cheering even they had no idea what happened.