AR Navigator

Inspiration

GPS can be accurate in outdoor environments, but people still get confused by those virtual paths and points shown on their device. Also, we need navigation for indoor environments, for instance, when shopping in a huge mall. With the rising of Augmented Reality (AR) technology, real-time, real-world-integrated navigations are made possible.

What it does

Using the camera view of the device, this application intends to render points of interest and navigation paths on the scene.

How we built it

We have used Unity as our main development platform and setup and integrated several SDKs such as Mapbox, Android SDK, Google AR Core, and Java development kit. Mapbox helped us get a general map of our location and then we built a virtual representation of walls and rooms as blocks in Unity. We had to make the walls invisible virtually but still manage to hide objects behind them so that the user can only see the object representing the destination when there are no walls blocking in between him/her and the destination.

Challenges we ran into

We initially wanted to assign geo-accurate points of interest, walls and walking paths as features to Mapbox's special layers. We started by taking the indoor layout image of the floor and then georeference it. Even though the layer features were assigned in Mapbox Studio, we were unable to render them in Unity and make them as the reference for the indoor points. We had to compromise by making a virtual representation manually in unity for reference to where the player is positioned. As a result, we made an assumption of the position where the user is at in the application.

Accomplishments that we're proud of

Building a virtual environment to navigate inside using AR.
Ability to see the 3D models we placed only when there are no blocking walls between them and the user.

What we learned

How to georeference a map by QGIS, upload to Mapbox Studio and understand its potential applications.
Learning and experimenting with AR.

What's next for AR Navigator

Render the Mapbox Layer of features we assigned in the map like points and paths in Unity.
Integrate with indoor position reference provider such as Bluetooth beacons.

Built With

Submitted to

JUNCTIONxKAUST 2018
- Winner Nanodegrees by Udacity

Created by

I worked on Unity3d and integrated the tools needed to produce this project. Researched and choose to use Mapbox + AR to implement the indoor navigation without the use of sensors. Made over 20 builds and tested them with the team on Android. Created virtual representation of walls, walkable paths, and matched them with the georeferenced indoor blue print. Learned a new shader technique to create the illusion of virtual walls called 'stencil mask'. I learned to capture our experience for the indoor navigation demo in the innovation building to showcase our work to the judges and made a simple demo to run it in an open space in the demo fair area which was in the Library building.

Faris AlShehri
MingchengTsai
Chen-Yu Ho
Mohamed Hamed
No one can describe himself, we only see what we aim to be, not what we actually are!