Bamboozled by the sheer food intake at the hackathons we've been to, we thought it would be interesting if we could make a product that would help us keep track of our food, and solve our 'ignorance in consumption'. The app couples the augmented reality technologies with the Google GCP and USDA APIs to provide a solution to this issue.
What it does
The application allows for a user to instantly identify and access important nutrition facts about a food they are about to consume. This includes macronutrients, as well as vitamins, minerals and other proximates. All of this data is accessible through a click of a button in an AR Environment, powered by SceneKit and iOS 11. In addition to this, the application provides statistical analysis on foods consumed on the daily, which allows users to digest the composition of their consumption.
How we built it
First, we quickly learned how the AR Kit worked, including using the 3D Coordinate Space, Scene Kit, and also scrapping images from the feed.
After this, we worked with GCP's Vision API Logo and Entity Detection in order to classify varieties of foods. With that data, that allowed us to make use of the public USDA Government API, giving us access to nutrition facts for very obscure foods (everything from pie to energy-drinks).
Lastly, we put everything together in an iOS app, and used UI Frameworks like Charts and Transition-Deck to make for a fluid experience.
Challenges we ran into
We found that AR Kit was quite challenging. It was difficult to use some of the UI Frameworks, as they were incompatible as a part of SceneKit. Other than this, we experimented with background threading to optimize the networking backend of the application.
Accomplishments that we're proud of
We found that the speed of the app was pretty great, as well as the analysis that it provided to the user. We also really enjoyed finding out more about the food we ate everyday, as well as the science behind calories and other nutrients.
What's next for CalARie
We would love to release CalARie on the App Store!