cleanse

Scanning foods, cleansing your plate and saving lives.

Comment

Inspiration

As members who are gluten-free, halal, and lactose-intolerant, along with family members having diabetes and high blood pressure, it's important to us that we know what ingredients are in our food and more importantly: what those ingredients mean. As you go further into an ingredients list, it becomes harder to understand what they mean, especially with preservatives and additives being a common part of foods you'd find at a grocery store. With the issue of interpretability, it's harder to understand the ingredients in your food, leading to harmful preservatives and foods that could complicate your health implications, without you even knowing. To solve these issues, we created our app, cleanse.

What it does

Our app, cleanse creates a personalized nutrition plan by scanning a barcode on any food product, it's "universal product code" (upc), and obtaining information on its health implications in context to you.

When entering the app initially, you enter your name, age, height, weight, and physical activity as identification information, which also calculates your BMI. In addition, cleanse intakes your current health implications along with your personal preferences. Then, you can scan a barcode of any food product, and it will pull up nutritional information, nutrition facts, suggestions for your health and a risk score (on a 100 scale) of the food you scanned in context of your health data.

The 100 point scale on health derives from an algorithm designed by us. We derived from our research that the biggest influences on health choices were implications and additives/preservatives. Doing so, we used a list of common implications like high blood pressure, diabetes, cardiovascular disease, pregnancies and more, as well as additives and preservatives from the CSPI Food Additive Safety Ratings, we created a scale of high risk, moderate risk, and low risk, putting context into the impacts of common foods that are harmful for people with implications. We first calculate a generalized score based off the nutrition facts, and then have multipliers that reflect hte impact of the preservatives on the overall rating.

Our algorithm uses Gemini to process the algorithm score, along with the patient information and nutrition facts, to give feedback and provide the user with standardized and simple language for any user to understand; where transparency is key.

The app will also keep a history of your scans in a separate section, allowing you to access the information easily, and at any time.

Finally, the app gives recommendations for future things to purchase in context of your account. Based off your previous scans, new suggestions will show up in the recommendations tab, allowing you to take the next steps in improving your health.

How we built it

We built the frontend using Expo and React Native, and the backend with Python, FastAPI, and LangChain to bridge the two stacks and manage LLM interactions. For data storage, we used SQLite to securely store personal health data locally, ensuring patient privacy. Using a computer vision model, we extract metadata from barcodes and cross-reference it with databases mapped to universal product codes to retrieve nutritional information. Our scoring pipeline then analyzes this data—flagging harmful preservatives, identifying dietary conflicts based on the user's health conditions, and evaluating overall nutritional quality. The algorithm dynamically adjusts a multiplier to deliver a personalized health score tailored to each individual’s needs.

Challenges we ran into

The biggest challenge we ran into was being able to universally identify ingredients with our algorithm. Ingredients were not standardized across our ingredients list, and what would be listed on actual food items. For example, someone with cardiovascular disease would be recommended not to eat processed meats, but the food products may say a specific meat like pork instead of the keyword "processed meats". To solve this challenge, we decided to use a LLM to give context instead of using keywords, allowing for ingredients to become universally standardized and accurate.

Accomplishments that we're proud of

  1. Building a barcode scanning app in less than 24 hours
  2. Being able to scan any food item and receive nutritional information on it

What we learned

We realized that working with FastAPI and React Native Expo for mobile apps is a lot more nuanced than we first thought. On the bright side, we now know way more than we expected about dietary restrictions and how they relate to different health conditions. On the not-so-bright side, branching four times off main taught us a lesson in Git pain we won’t forget anytime soon. Also, running all our Expo tests on mobile hotspots? Probably not our smartest move—but hey, lesson learned.

What's next for cleanse

  1. Scale to more products - Our barcode scanning interface is able to read any upc code, meaning it can be leveraged and scaled to more products in the future that still impact health.
  2. Solve the issue with variability with ingredients - Find more methods outside using an LLM call to be able to give context across different ingredients through our algorithm

Built With

Share this project:

Updates