Shelf Aware

Simplify Shopping. Amplify Impact

Comment

Shelf-Aware

Inspiration

Consumers today face significant challenges in finding products that align with their values. The key problems we're addressing:

  • Difficulty in finding products that match multiple criteria (cost, health, environmental impact, reliability)
  • Time-consuming process of assessing these factors, especially environmental impact
  • Gap between sustainability-focused companies and their target audience

What it does

Shelf Aware empowers consumers to make informed, value-driven decisions by analyzing products based on multiple factors. Our platform features:

  • Product detection via webcam or shopping links (e.g., Amazon)
  • Comprehensive analysis providing health, sustainability, and reliability metrics
  • Personalized alternative product suggestions
  • Partnership system with eco-conscious brands for recommendations and rewards

How Shelf Aware Works

1. Product Lookup

Users scan products with a webcam or paste shopping links.

2. Data Analysis

Shelf Aware uses computer vision or web scrapers to detect the product, and scrape and aggregate data from trusted sources (like OpenFoodFacts database, corporate sustainability reports, etc) to generate health, sustainability, and reliability ratings.

3. Customized Recommendations

Users receive alternative product suggestions tailored to their priorities (e.g., cost, sustainability) which they set when they create an account. Recommendations often include sustainable brands due to our partnerships with them.

4. Incentivized Choices

Users earn points for choosing the eco-friendly products we suggest. These points can be redeemed for partner-provided coupons, creating a gamified feedback loop that rewards sustainable purchasing habits.

Our recommendations create a positive feedback loop: users choose eco-friendly products, earn points for sustainability, and redeem partner coupons to make even more environmentally conscious purchases.

How We Built It

Frontend Development

  • React + Vite for rapid development and optimal performance
  • NextUI components for sleek, accessible interface
  • Custom hooks for state management and API integration
  • Responsive design with Tailwind CSS

Computer Vision Pipeline

  • Custom-trained TensorFlow model for product recognition
  • EasyOCR for text extraction from product images
  • OpenCV for image preprocessing and enhancement
  • YOLO v8 for object detection and segmentation
  • Containerized ML pipeline for scalability

AI/ML Integration

  • GPT-4 API + data aggregation from sources like OpenFoodFacts database, corporate sustainability reports, etc using Perplexity Web for product analysis and recommendations
  • Custom embeddings for product similarity matching
  • Fine-tuned models for sustainability scoring
  • Real-time inference optimization

Backend Architecture

  • Flask REST API with async support
  • Modular service architecture for scalability
  • Rate limiting and request validation
  • Comprehensive error handling and logging

Database & Storage 📦

  • Supabase for real-time data management
  • Efficient schema design for quick queries

Integration Points 🔄

  • OpenAI API for natural language processing

Challenges we ran into

  • Developing accurate product detection systems - developing an efficient OCR (Optical Character Recognition) model for product detection was challenging, and we spent a lot of time optimizing our model to work within a web app.
  • Creating an effective recommendation algorithm was challenging, and it was hard to aggregate data from multiple sources in a meaningful way. We also struggled with integrating this with Perplexity's web searching LLM that was hallucinating.

Accomplishments that we're proud of

  • Created a functional proof of concept that demonstrates our core vision
  • Building a business case for our product
  • Building an OCR model that detects products successfully in real-time live video feed.
  • Being able to scrape data from multiple sites and aggregate it

What we learned

  • Use of tech stack such as YOLO, an implementation of object classification in real time for our OCR model.
  • The importance of balancing multiple user priorities in decision-making
  • How to create engaging feedback loops for sustainable behavior
  • The complexity of gathering and analyzing product sustainability data

What's next for Shelf Aware

What we have built here is a functional proof of concept rather than a finished product. Planned improvements include:

  • Advanced personalization algorithms
  • Enhanced product detection capabilities
  • Implementation of user reviews
  • Market studies to better understand gaps between consumer needs and sustainable product availability
  • Further development of our partner network

Impact

Shelf Aware simplifies informed decision-making, promotes sustainable consumption, and helps eco-conscious brands thrive—all while turning environmental responsibility into an engaging, rewarding experience.

Built With

Share this project:

Updates