Prototype - Dashboard
Prototype - Receipt/label scanning
Prototype - OCR result
Prototype - Items After Nova Classification

👀 Inspiration

The inspiration behind NutriSnap arises from the prevalent misleading health information, often propagated by social media. There's a crucial need to counteract this and provide users with a comprehensive understanding of nutrition. Additionally, NutriSnap is motivated by the gap between biotech research and practical applications. Recent advancements, like the NOVA classification system, shed light on food processing's health impact, yet this knowledge remains inaccessible. NutriSnap addresses this by leveraging technology to deliver accessible health information, empowering users to make informed dietary choices and promoting holistic health understanding.

🤖 What it does

NutriSnap allows users to scan grocery receipts or food labels to retrieve detailed health information about the products they've purchased or are considering consuming. The platform leverages a combination of technologies, including OCR (Optical Character Recognition) for extracting ingredients, nutrition facts, and allergy information from labels, and a NOVA classification model to predict the processed level of the food items. This comprehensive approach aims to empower users to make informed decisions about their dietary choices, considering not only calorie and nutrient content but also the level of food processing.

👩🏻‍💻 How we built it

We built NutriSnap using a variety of tools and technologies. The frontend was developed using Vue.js to create an intuitive and user-friendly interface for scanning receipts and labels. For the backend, we utilized Python Flask to handle the OCR processing and integrate with external APIs, such as the OpenAI API for additional data retrieval. The NOVA classification model was implemented using a random forest tree classifier, trained on a manually identified dataset of NOVA classes comprising >200 data points.

❗️ Challenges we ran into

Throughout development, we grappled with challenges testing our problem-solving skills and perseverance. Our passion for biotech clashed with scarce or high-dimensional data, ethical concerns, and immature technologies, hindering effective utilization. Post-NutriSnap decision, we tackled issues spanning basic web development, OCR data processing, and NOVA classification dataset collection and cleaning. Optimizing the OCR model for diverse label formats and languages demanded meticulous refinement, while training the NOVA classifier with limited data posed accuracy challenges within hackathon timeframes. Despite these hurdles, our collaborative spirit and expertise drove progress, reflecting our commitment to overcoming obstacles and advancing Nutrisnap's development.

🌟 Accomplishments that we're proud of

Despite the challenges we faced, we're proud of several accomplishments achieved during the development of NutriSnap. Firstly, successfully integrating receipt scanning and label OCR functionality into the platform allowed us to provide users with a seamless experience for accessing health information about their food purchases. Additionally, conducting thorough research on the bio side and developing and implementing the NOVA classification model, even with a relatively small training dataset, demonstrated our ability to apply machine learning techniques to address real-world problems in the health and nutrition domain. Overall, we're proud of the holistic approach we've taken in creating NutriSnap and the potential it has to positively impact users' health and well-being.

📝 What we learned

Throughout NutriSnap's development, we gained practical experience in utilizing OCR technology and integrating machine learning models into web applications. Our understanding of nutrition science deepened, emphasizing the significance of food processing in health effects. Effective team collaboration and time management in a fast-paced hackathon environment honed valuable skills for future projects. NutriSnap provided a rich learning experience, offering insights into the intersection of technology and health.