The Hololens adds an additional dimension to our 3d physical world. We think it inspires people better than any other digital devices cause it allows user to get more understandable information about the surroundings without limiting the line of sight within a tiny digital screen. Based on the device/platform we have passion about we want to help people develope a healthy, sustainable life style. It is not only of personal advantage for a positive life, but it also contributes to an environment-friendly society, for example, by controling the amount of comsuption in food of the user, the waste is reduced.
What it does
It keeps track of life routines of the user, including his/her health condition and exercise habit, and documents grocery items the user buys for meal. Based on the ingredients the Hololens find avaliable in the fridge via image recognition, or the data provided by some Smart Home devices, our app will find recipes for a meal that in accordance with the user's diet habit and in support of user's health condition and excersise habit through nutrition suggestion. A ranking system would be built based on user's extra input (e.g. personal preference) and suggestions from Elena (the health-assistant chat-bot built by University of St. Gallen). For a mature end-product we thought about, a QR-code would be put on the table in the kitchen area in the house of the user. The Hololens recognize the code automatically when the camera captures it. If the local time is around the meal time, our app starts automatically and directly provide the user 3 choices of recipes taken from the top 3 by the ranking system. Some extra information, e.g. cooking time, contained Calorie, etc, would be provided and displayed, which is personalised by the user in the settings of the app. After one recipe is chosen the Hololens displays 3d virtual objects of ingredients needed by the recipe on the table. Through animations the amount of each ingredient needed and operations like cutting or peeling required for every step would be displayed in an understandable and interactive way. At the same time Elena would provide health advices via voice, which is more user-friendly in a mixed reality situation.
How I built it
We built the app through Unity with MRTK.
Challenges I ran into
- With a successful build of our code, unfortunately we could not test it on the real device Hololens2 provided by University of St. Gallen. The reason is that to enter the development mode of the device we need authentification from the IT department of the University, but the communication is sadly impossible on the weekend.
- Virtual and on-site hacking combined has been challenging. From the communication side we were facing problems like low sound quality and loud enviromental noise in Zoom calls. It led to low efficiency of exchanging thoughts and ideas, especially in our case that we do not know our remote teammates at all before the event.
- The Hololens2 is a brand new and unique product with very limited popularity, so that online there's neither many documentations nor various discussions regarding development problem. We are short of help when we ran into problems.
Accomplishments that I'm proud of
- We are one of the only two teams at HackZurich that have the guts to claim a brand new challenge on a cutting-edge device Hololens2 that even not many people have heard of it. We are an adventurous team!
- We came up with a scenarior that fits our original goal of hacking (we could try something new, and the project is equipped with some practical value). For the app we designed a good architecture for the development, even though half of our team members do not study computer science. We split the job well so that everyone got technical things to investigate, as well as have fun in the whole event.
What I learned
We learned some basics about app development for Hololens 2. We learned how to dismantle a quite big goal into small tasks that are achievable.
What's next for HaloFit
If we could have a testable device, test our current build must be the first step. Then based on the performance we might need to debug and improve data flow efficiency. Then we have to replace all our current fake data with practical API. Finally we would realize all other functions we originally designed for the scenario.