AR Kiosk

Inspiration

As an indie game developer from Kenya, marketing my games has always been challenging. Major gamedev conventions such as PAX and E3 are never hosted in Africa. African-made games are almost non-existent in the market. So, if I can't afford to make it to PAX, why not build my booth and share it with the world? Hence BINGIVISION: AR Kiosk (or simply AR Kiosk).

What it does

AR Kiosk is a virtual expo solution in augmented reality that helps save on logistics costs for exhibitors and brands. It features:

a customized, 3D AI chatbot trained on brand data to help customers understand the brand through conversation.
users can view products in 3D and interact with objects by zooming in and out and 360 degrees rotation;
Video player to play trailers, demos, pitches, etc.
Slide decks.

How we built it

The project was created with Unity, C#, AR Core, Blender, and Adobe Illustrator. Using Unity engine and C# as the main medium, the AR Core package was imported to create an AR scene. Blender was then used to model a booth and 3D renders of products. I created a state machine architecture for the system. Reception State, Slides State, Video State, and ProductView State. In the Reception State, we use Inworld.ai API to create a customized AI chatbot. The Slides State image canvas shows images that switch on player interaction. The feature was built from scratch in C#. Video State displays a series of videos by rendering them on a raster image on the canvas. Product State is built from scratch using C#. A 3D model is placed in front of the camera and Lerp() function is used to manipulate the object via scale and rotation transform.

Challenges we ran into

Getting proper 3D renders of the map in Cesium and Geospatial Creator is a challenge. Much of the Kenya map is flat. The initial plan was to geo-tag specific locations and place different booths in each but it became a challenge to find the correct height placement. The fallback solution was therefore to cache some data and use a single booth model and then switch booth information when the player is within a specified radius of the geo-tag.