EcoCart AI

Dashboard
Supermarket 3D
Smart Grocery List
Leaderboard

HackDuke 2026 Sustainability, Gemini, Auth0 Tracks

Inspiration

Every year, the average American household generates over 8.1 metric tons of CO₂ from food alone — yet most people have no idea which items in their grocery cart are the biggest offenders. We kept asking ourselves, what if the receipt you throw away after every grocery run could actually teach you something?

At HackDuke 2026, we wanted to build something that turns a mundane habit — grocery shopping — into an act of environmental awareness. Not by guilt-tripping people into eating differently, but by showing them that small, practical brand-level swaps can make a measurable difference. You don't have to stop buying chicken — you just have to know which chicken to buy.

That idea became EcoCart AI, a platform that meets shoppers where they already are and gives them the tools to make greener choices without changing their lifestyle.

What it does

EcoCart AI is an AI-powered grocery sustainability platform with five core features:

Receipt Scanning & Carbon Estimation Users upload a photo of their grocery receipt, which is processed by our OCR pipeline to extract data, including product name, brand, quantity, and price. This data is then passed to Gemini AI, which structures the data and performs product-level analysis by mapping each item to its underlying ingredients and categorizing them for emissions relevance. Based on this classification, we estimate the carbon impact of each product by applying standardized emissions factors (kg CO₂e) sourced from the Climatiq API. For a typical receipt of around 15 items, the system generates a detailed, itemized carbon footprint breakdown in under 20 seconds.

Smart Swap Recommendations For the highest-impact items on a receipt, EcoCart AI recommends realistic brand-for-brand alternatives, not "replace your steak with tofu." If you bought Tyson chicken breast, we'll suggest Perdue Harvestland Organic. If you bought Horizon milk, we'll suggest Organic Valley from a local cooperative. Every swap comes with a specific CO₂ savings estimate and a reason why it's better.

3D Supermarket Navigation A full Three.js-powered 3D model of a real grocery store, built from an actual floorplan. Users can select items from their receipt and generate an optimized walking route through the store using a zone-based sweep algorithm that minimizes backtracking. Eco-friendly products glow green on the shelves.

AI Sustainability Coach A persistent Gemini-powered chat sidebar where users can ask anything about their purchase history, get personalized tips, or compare the carbon impact of different products. The AI has full context of the user's shopping patterns.

Gamification & Leaderboard Points, streaks, badges, and a global leaderboard that turns carbon reduction into a community challenge. Users earn points for scanning receipts, accepting swaps, and maintaining daily engagement streaks.

How we built it

Backend: Python Flask with Flask-SQLAlchemy and SQLite. Auth0 handles authentication via OpenID Connect. The Gemini 2.5 Flash API powers both the vision-based receipt parsing (structured JSON output with response_mime_type) and the conversational AI coach.

Carbon Estimation Pipeline: We built a two-layer estimation system. First, Gemini extracts items and estimates CO₂e using LCA-informed prompting. Second, a validation layer cross-checks against a lookup table of ~100 food categories with published emission factors (sourced from Our World in Data and peer-reviewed lifecycle analyses). If Gemini's estimate diverges by more than $5\times$ from the lookup, the lookup value takes priority. The carbon cost per item follows:

$$\text{CO}_2\text{e} = \text{emission factor (kg/kg)} \times \text{quantity} \times \text{unit conversion factor}$$

Brand Assignment: A 200+ entry keyword-matching system ensures every receipt item gets a real-world brand, even when the receipt abbreviates ("GV" → Great Value) or omits the brand entirely. The fallback maps product keywords to the most common US grocery brand for that item.

3D Store: Three.js with OrbitControls, loaded as an ES module via importmap, no bundler needed. The store geometry is built entirely from primitives (BoxGeometry, PlaneGeometry, Sprites) based on a real supermarket floorplan we digitized into zone coordinates. Product cubes are color-coded by carbon impact and respond to hover/click with detailed popups.

Routing Algorithm: Instead of naive nearest-neighbor (which causes backtracking), we implemented a zone-based sweep that groups items by department, visits departments in a counter-clockwise perimeter loop, then handles center aisles front-to-back. Paths follow actual walkable corridors to avoid cutting through shelves.

Frontend: Server-rendered Jinja2 templates with vanilla JavaScript for SPA-like view switching. No React, no build step, just clean HTML/CSS/JS with Lucide icons and the Outfit typeface.

Deployment: Dockerized with Gunicorn, one-command docker-compose up.

Challenges we ran into

Gemini hallucinating carbon values. Early on, Gemini would claim a banana had 15 kg CO₂e or that rice was carbon-negative. We solved this with the two-layer validation system, trust Gemini for item extraction, but verify its CO₂ numbers against known emission factors.

Receipt parsing is messy. Real grocery receipts abbreviate everything ("HNY CRS WHL WH" = Honey Crush Whole Wheat), use store-specific codes, and have wildly inconsistent formatting. We iterated on the Gemini prompt extensively, adding explicit instructions for abbreviation expansion and brand inference.

Three.js in a non-bundled Flask app. Getting ES module imports to work without webpack or Vite required use of importmaps and separating the 3D code into its own module file. Initializing the renderer while the container div was still display:none produced a 0×0 canvas with zero errors.

Making swaps actually useful. Our first version suggested replacing beef with lentils and milk with oat milk, technically lower carbon, but not what any real shopper would do. We rewrote the swap system with strict same-product rules and a validation layer that rejects cross-category substitutions.

SQLite groupby fragility. Python's itertools.groupby breaks if the data isn't pre-sorted by the grouping key. Our receipt history was ordered by timestamp, not receipt_id, causing receipts to render as duplicated fragments. We replaced it with dict-based grouping.

Accomplishments that we're proud of

End-to-end receipt → insight pipeline in under 10 seconds: Upload a photo, get itemized carbon data, brand-level swap recommendations, and a 3D store route, all from one image.
A 3D supermarket built from a real floorplan: Not a generic box, our Three.js store matches a grocery layout with accurate department zones, shelf rows, and walkable corridors.
Swaps that people would actually use: Our validation system ensures every recommendation is a realistic brand-for-brand alternative, not a lifestyle lecture.
200+ brand fallback mappings: Every single receipt item gets a real brand, even on the most abbreviated receipt printouts.
Zone-sweep routing with zero backtracking: Our path algorithm visits each department exactly once in a logical loop.
Full gamification loop: Points, streaks, 7 earnable badges, levels from Seedling to Guardian, and a 20-person leaderboard that makes carbon tracking feel like a game.

What we learned

Prompt engineering is half the product. The difference between useful and useless Gemini output came down to extremely specific prompting, enumerating exact JSON schemas, giving good/bad examples, and constraining the output space.
AI needs guardrails, not just instructions. Even with a perfect prompt, Gemini occasionally returns nonsensical values. The validation layer taught us that production AI features need deterministic fallbacks.
Three.js without a bundler is possible but tricky. Importmaps are underrated for hackathon projects, they let you use modern ES modules without any build tooling.
Gamification changes behavior. Even with fake leaderboard data during development, seeing a streak counter and badge progress made us want to upload more receipts. The psychology works.
Simple tech stacks ship faster. Flask + SQLite + vanilla JS meant zero time fighting build tools and all time building features.

What's next for EcoCart AI

Real store integration: Partner with grocery chains to use their actual planogram data for the 3D store, showing real-time inventory and pricing alongside carbon data.
Barcode scanning: Supplement receipt parsing with in-store barcode scanning for real-time carbon lookups while shopping.
Cumulative impact dashboard: Show users their month-over-month and year-over-year carbon trajectory with projections, "At this rate, you'll save 120 kg CO₂ this year."
Social features: Share receipts with friends, create household teams, and compete in community challenges ("Chapel Hill's Greenest Grocery Shoppers").
Multi-store 3D models: Let users select their local store and get a personalized 3D navigation experience.
Carbon offset marketplace: Connect high-impact purchases directly to verified carbon offset programs, letting
Rewards: Allow users to redeem rewards and discounts based on how many points they've earned. This directly incentives users to lower their carbon footprint. users neutralize their grocery footprint with one click.