BilkAAR

Lens icon

Inspiration

It’s squirrels!!! We all really love animals and we love winter. There wasn’t a lot of snow in Berlin this year (yet) so our best take on getting some proper snowball fight action was to make an AR Lens with this function. We all really love action games and we really enjoy “random” genre games, such as the famous Goat Simulator.

We would never think about hurting animals in the real world. Neither would we make it appear like virtual animals are hurt. The squirrels in BilkAAR are not evil, they are just sneaky and they are stealing presents. They are on a service for a green-skinned anti-Christmas villain and they must be stopped! A hit with a snowball only slightly disorients a squirrel and causes it to drop the present and then soar to the sky on a jetpack. The name for the game is made of “bilka” – Ukrainian for squirrel and AAR – the abbreviation for Active Augmented Reality – a term forwARdgame has coined to name the games where the controls or a significant part of the playing is done or encouraged to be done by physically moving in the real world.

What it does

It’s an active AR game where you need to throw a snowball and try to hit a squirrel with it. The squirrel then will drop the present and you need to approach the present and grab it with your physical hand. The game lasts 4 minutes and then you can try and improve your caught presents score. There is an easter egg -- discover it when you switch to the front camera.

How we built it – tech stack

Presents collecting: Hand tracking + gesture recognition + Hand occlusion Placing the tree and presents falling: World Mesh Tween for animation transition Physics + colliders for snowball throwing and hitting Custom solution for pathfinding for the squirrels to prevent them from bumping into each other and the tree branches Persistent storage system for scores Remote access – to store sound effects Front facing camera – Head tracking and mouth open -- Our awesome easter egg in the lens

Challenges we ran into

We had a few issues; first was to figure out the rotation of the squirrels – they need to turn towards the direction of their leap. A really weird problem is with adding an object inside the hierarchy – we expect it to be in 0,0,0 local position and 0 local rotation but it actually adds in random coordinates and random location within the parent. Additional trouble was with LookAt Offset Rotation, because on the scene objects have euler angles and in code it’s a normalised vector. There are issues with VFX – when moving the parent the child’s position doesn't change and instantiating the second VFX removes the first. Physics add force disregards the mass of an object.

In Unity there is OnTriggerEnter/Stay/Exit and OnCollisionEnter/Stay/Exit In Lens Studio there is OnOverlapEnter/Stay/Exit and OnCollisionEnter/Stay/Exit, while building the project we tried to use this feature to detect the hit with the snowball. The Overlap showed a really bad result – many times it just wouldn’t notify about the hit of the object with the snowball. So the better solution was to Collision and then registration of a hit was close to 100%.

Accomplishments that we're proud of

We were very happy to find a neat trick to solve the physics issue – by changing the physics simulation rate. We are exceptionally proud of the custom pathfinding solution. We made sure squirrels don't run into each other and avoid branches. We also made an effort to sync the spawning of the squirrels, putting the rate in order, making sure there are too many and not too little at any given time.

On top of all we are very happy with how dynamic and fun the game came out to be. It really invites the players to be physically active for a little while. The game is both challenging and rewarding – we are very happy we managed to balance these two game aspects.

What we learned

Work with physics and collision was new to us and this project was an excellent sandbox for us to learn more about physics in Lens Studio. And advanced tween abilities were on our TODO list for a long time and we are so happy we had a great reason to dig it with BilkAAR.

What's next for BilkAAR

Of course the absolute must is the multiplayer -- collocated and remote! Our rule in AR game development is that if it's really fun as a single player -- it's gonna be massive as a multiplayer. We are planning to add more game elements as difficulty levels. And we are planning to add buffs, such as hedgehogs, so that if the player has one – the hedgehog can collect some of the presents that squirrels dropped.