Immersive technologies including AR and VR have already been used in gaming and communication. Everybody loves discovering new places and exploring landmarks. However current technologies including GPS location fails us in the cities with high rise. Our goal was to bridge this gap using the latest AR/VR technologies. We have used scanning the public station name or remarkable landmarks present around to identify your location.
By pasting specific QR Codes in different Jelbi Stations or in interesting landmarks... Users can quickly get real time information about proximity of Jelbi Partners services e.g bikes, bus proximity, rental car or battery location. Inmersive technologies such as AR allow you display all available information around you without clottering the space, speeding up the time it takes for users to look up availabilty of the services.
What it does
It allows you to take picture of the public transport station or landmarks around to identify your location.
How we built it
For the microsoft hololens, we used mixed-reality toolkit on Unity Engine. 3d models were created in Blender and Gimp. We deployed a REST-API on AWS that calls different Jelbi-Partner APIs and gathers data to send back to the hololens or AR application. Using hand gestures on the Hololens, HTTP requests get triggered, calling the REST-API and retrieving live data.
Challenges we ran into
We had no idea how to integrate REST API calls and other services in node and python in Unity and Microsoft Hololens. We looked at the possibility to use Azure cognitive services. Finally we figured out a way to make an HTTP request from Unity.Unity Engine along with Microsoft Visual studio and mixed-reality toolkit are very bulky and operate only on MSFT windows. So not everyone had this environment to prototype rapidly.
Accomplishments that We are proud of
By touching on a floating BIKE icon on the Hololens field of view... We call the next_bike API and retrieve data about the location of the available bikes. By computing the distance of all these bikes to our location we found the distance and coordinates of the nearest bike.
- But interestingly, by giving these coordinates into Google Street View API, we can get a picture of the closest snapshot taken by google near this location and display in the hololens. Which helps the user know how the location/street the bike is in, looks like.
The 3d model creation and visualizing in hololens was amazing. The immersiveness hololens provides is best to any other technologies.
What we learned
Everything from creating 3d models. using Unity and integrating other services in unity and how to trigger events inside of the hololens. Making the outer world accesible with requests.
What's next for RELBI AR
We are going to keep working and integrating novel immersive technologies to make mobility better and easier for all the explorers and wanderers out there.