With the ability to experience augmented reality and the growing capability of computer vision, we are determined to combine two of the newest technologies into an app that will transform information gathering in the real life.
What it does
Using HoloLens as a direct interaction with the user's environment, information about the surroundings is sent to the Microsoft Azure server as series of pictures. The Cognitive Services API is then used to recognize and analyze the visual input, which will automatically be displayed on the HoloLens.
How we built it
The front-end of HoloMetrix is built with Unity and Visual Studio using c#. Microsoft Azure and node.js, on the other side, are used for the creation of the middleware.
Challenges we ran into
Monster and lots of them. But most importantly, setting up the correct HoloLens development environment and learning to use Unity and Visual Studio.
Accomplishments that we're proud of
Despite having to code in Unity and Visual Studio for the first time and the struggles along the way, we are able to complete a HoloLens app within a short time frame. Good job team!!!
What we learned
We explored the opportunity of hacking a HoloLens while incorporating many aspects of the Cognitive Services API.
What's next for HoloMetrix
Currently, we have integrated object and face recognition. To expand on this project, other APIs, such as Walmart Open API, can be used as a tool to gather more information.