BiteScan

Inspiration

Our commitment to eating healthy kept us motivated throughout this project. We developed a web app that allows users to take a picture of a snack’s ingredients, and the app analyzes whether the snack is healthy or not.

What it does

The app captures an image of the nutritional facts of a snack or beverage, extracts the information, and analyzes it using OpenAI’s API. It then provides meaningful insights and recommendations on whether the item is suitable for consumption.

How we built it

Frontend: We used Next.js for its fast rendering and efficient routing. Backend: We integrated Spring Boot to ensure seamless API interactions. Firebase and Google Cloud Vision were used for image storage and text extraction. AI Model: The backend leverages AI to process images, extract ingredients, and analyze their nutritional value. We used OpenAI’s ChatGPT-4o Mini model, fine-tuning it to function as an experienced fitness instructor.

Challenges we ran into

1) Choosing the Right OCR & Storage Solution: Initially, we opted for a native relational database for image storage and planned to use Tesseract for OCR extraction. However, we later realized that combining Firebase with Google Cloud Vision provided a much more optimized solution. 2) Browser Security Limitations: Since our website runs on an HTTP protocol instead of HTTPS, modern mobile browsers restrict access to sensitive features like cameras. Unlike permission-based restrictions, this is a security policy that cannot be bypassed. As a result, we temporarily pivoted to a desktop-only approach, where webcams can still be accessed over HTTP.

Accomplishments that we're proud of

1) Successfully integrating cutting-edge technologies like Firebase and Google Cloud Vision into our application. 2) Configuring ChatGPT-4o to act strictly as a fitness expert, limiting its responses to the assigned role rather than functioning as a general AI tool.

What we learned

Time Management – Balancing development, testing, and iteration efficiently. Utilizing AI Models – Understanding how to fine-tune AI responses for a specific use case. Google Cloud Services – Learning how to leverage cloud-based tools for image processing and data storage. Firebase APIs – Gaining hands-on experience in integrating Firebase for seamless backend functionality.