Our team wanted to work with VR/AR in a way as to benefit society. More specifically, we wanted to provide users with a convenient way to track their daily calorie and macronutrient intake. We discovered that many people found it tedious to look up nutrition information or even look at nutrition labels, so we condensed the macronutrient information into an easy-to-read bar that shows up when the user's camera recognizes the food.
What it does
NutriBAR uses your iOS device's camera to scan its frame for recognized foods and returns a brief, condensed label containing serving size, calories, and macronutrient breakdown. If the device instead sees a two-dimensional image of a recognized food, it will display a corresponding three-dimensional model as well.
How we built it
We constructed a database of foods with their corresponding macronutrient information and used Unity's image recognition software and the Vuforia plugin to scan its surroundings and detect any recognized foods. We used Tinkercad to build the 3D models for detected 2D images.
Challenges we ran into
Some obstacles we encountered included having to map both a 3D model and a 2D label to targeted images since we found that the matching was often unstable, and the 2D image was tricky to map onto a 3D plane. However, we were able to alleviate the problem by passing in higher quality photos into our database so that Unity could recognize them more easily (since we found Unity's image recognition algorithm to be limited in its scope).
Accomplishments that we're proud of
This was our first time working with augmented reality software, and we are very proud to have been able to complete a functioning project in a limited time span. We're also glad to have been able to contribute to a growing field, and are excited to continue to develop.
What we learned
We learned how to map both two-dimensional and three-dimensional objects onto real-world items. We also discovered that there huge possibilities for AR in a wide range of fields and are excited to see what the future of VR/AR holds.
What's next for NutriBAR
In the future, we would really like to use a more developed image recognition algorithm, since we found that Unity's software was not as comprehensive as we would have hoped. With a more robust algorithm, our app could recognize virtually any dish and be able to query a nutrition API and construct the virtual label with the basic nutrient breakdown.