While nutrition labels appear to be transparent with the comprehensive breakdown they provide regarding key nutritional information, the practice of deliberately changing the calorie amount can have a significant impact on the health of consumers, with some labels having up to 70% error. Caloreye aims to solve this by introducing true transparency to the user, with a complete breakdown of the correct calorie amount based on the scanned nutrition label.

How it Works

When the user opens the app, they are greeted by the scan button, which, when pressed, the user is prompted to take an image of a nutrition label that they would like to have analyzed. From there, the image is sent to Google’s Firebase Cloud Storage, where it is stored in a storage bucket accessible via its URI. The application then accesses and downloads the image URI. The image URI, along with the image detection method, are supplied as inputs to the Google Vision API, which uses optical character recognition technology to detect text within the image and return a list of associated characters and words. The numerical information pertinent to calorie calculation is subsequently separated and displayed for the user in a table, along with an insightful breakdown of each macro’s contribution to the true calorie count.

Challenges Faced

Connecting to the Google Vision API was undoubtedly the most difficult part of the project. Namely, we ran into issues upon passing in the downloaded image URI from the Google Cloud Storage when making the request to the API. One issue was that the API could not access the downloaded URI returned on image upload, and required passing the actual cloud storage bucket location of the image. Also, we ran into permission issues when getting the API to try and access the bucket.

Learning Opportunities

Overall, Caloreye provided us with an immense learning process. Principally, we expanded our skills in React Native, learning how to access the camera in integration with the Google Vision API to interpret and send back data in a meaningful way.

Share this project: