SmartPick — Devpost Submission

Inspiration

The #1 reason products get returned on Amazon isn't defects — it's unmet expectations. Shoppers guess, buy wrong, and return. And that decision costs more than money.

30% of online purchases are returned — the highest in retail history
5 billion kg of returned goods hit landfills every year
24 million metric tons of CO₂ come from return logistics annually — equivalent to 5 million cars

We kept asking ourselves: what if the problem wasn't buyer regret, but an information gap? Shoppers don't get a defective product — they get the wrong product. They couldn't visualize it, couldn't understand how it fit their specific use case, and couldn't compare what actually mattered to them.

Amazon is the largest e-commerce platform on earth, which means it also processes more returns than anyone on the planet. We saw an opportunity: if we could help shoppers make more confident decisions before buying, we could reduce returns at scale — and the environmental cost that comes with every single one.

That's SmartPick. We don't tell you what to buy. We illuminate the decision so you can make the right one yourself.

What it does

SmartPick is a three-step AI-powered product comparison tool that lives alongside Amazon.

Step 1 — Pick products Select 2–4 products from the catalog, or paste any Amazon URL. The extension auto-imports the product directly from the listing page.

Step 2 — Answer 5 questions Claude (Anthropic's AI) generates category-specific preference questions drawn from real reviewer concerns — not generic dropdowns. Commuting headphones user? You get different questions than someone working from home. Your answers reshape the entire dashboard around your priorities.

Step 3 — See the full picture A rich 8-tab comparison dashboard covers every angle:

Tab	What it shows
Eco Impact	Return risk score, CO₂ cost of a wrong decision, SHAP feature breakdown
Overview	Radar chart across 6 dimensions (value, quality, reliability, longevity, sentiment, sustainability)
Specs	Side-by-side feature matrix with per-spec winner highlights
Price & Value	90-day price history — know if today is actually a good deal
Reviews	Full star distribution + sentiment analysis, not just an average
Reliability	Historical failure-rate timeline (3mo / 6mo / 1yr+)
3D Models	Interactive 3D model generated from the product image — rotate, inspect, zoom
AI Recommendation	Claude synthesizes your wizard answers, ML scores, and spec data into a personalized verdict

Chrome Extension The extension auto-injects a return risk badge on every Amazon product page — green/amber/red, right next to the buy button. No tab switching. Click "Compare on SmartPick" and the product imports automatically.

Personal Impact Dashboard After signing in with Auth0, users track their own CO₂ avoided, returns prevented, and comparisons made — making sustainability something you can see and earn.

How we built it

Frontend

Next.js 16.2.2 (App Router) + React 19 + TypeScript
Tailwind CSS v4 for styling, Recharts for all data visualizations
Three.js + @react-three/fiber for the 3D model viewer
Custom CSS IntersectionObserver scroll-reveal animations on the landing page

AI Layer

Anthropic Claude claude-sonnet-4-6 — generates category-specific preference questions, writes the key-differences summary, and produces the final personalized recommendation
The prompt is structured so Claude knows the user's use case from the wizard answers and weighs the ML risk scores in its output

Machine Learning

XGBoost regressor trained on 8,000 synthetic samples across 11 features: price, expected lifespan, materials quality, brand tier, Amazon review sentiment, weight, packaging recyclability, shipping origin, and 2 intentionally irrelevant features (release year, product ID hash) to demonstrate that SHAP correctly ignores noise
SHAP (SHapley Additive exPlanations, from game theory) layered on every prediction — each score comes with a waterfall chart showing exactly which features pushed return risk up or down
Served via FastAPI microservice on port 8765

3D Generation Pipeline (three-tier fallback)

Tripo3D API — cloud-based 3D mesh from product image (~30s)
Meshy AI API — alternative cloud provider fallback
SF3D (Stability AI) — local open-source 3D reconstruction, ~2s after warm-up on GPU

FastAPI microservice on port 8001 manages the fallback logic and caches GLB files to avoid re-generation.

Authentication

Auth0 v4 (@auth0/nextjs-auth0 ^4.16.1) — secure session via HTTP-only cookies
Personal impact data stored in localStorage keyed by Auth0 user ID

Chrome Extension

Manifest V3 compliant
Content script auto-detects Amazon product pages, extracts ASIN, title, price, brand, image, rating, and review count
Injects return risk badge and floating panel with eco data
Popup supports queuing a second product and launching a full comparison

Challenges we ran into

Auth0 v3 → v4 migration The installed version was v4.16.1, which has a completely different API from v3. handleAuth, UserProvider, and withApiAuthRequired were all gone. Routes changed from /api/auth/... to /auth/..., and AUTH0_ISSUER_BASE_URL became AUTH0_DOMAIN + APP_BASE_URL. We went through multiple rounds of build failures to arrive at the correct pattern: Auth0Client singleton + auth0.middleware in Edge-compatible middleware.ts.

Next.js Edge Runtime + Auth0 auth0.middleware cannot be instantiated via a server-only module import when running in Next.js Edge Runtime. Exporting the bound method (export default auth0.middleware) strips the this context, causing Cannot read properties of undefined (reading 'authClient') at runtime. The fix was wrapping it: export default function middleware(req) { return auth0.middleware(req) }.

Amazon image CDN blocking Manufacturer websites (Sony, Bose, Brooks) block hotlinked images with 403/503 responses. We audited all product image URLs using PowerShell HEAD requests and replaced them with verified m.media-amazon.com CDN URLs. We also had a case where a previous session accidentally assigned Sony WH-1000XM5's image ID to the Winix 5500-2 — found and fixed by fetching actual Amazon product page HTML to extract the correct image ID.

Cloudflare tunnel WebSocket failures Running behind a Cloudflare tunnel for external demo access broke Next.js HMR WebSocket connections, causing an error overlay that intercepted click events — which is why the light/dark mode toggle appeared broken externally. Fixed by adding allowedDevOrigins: ['*.trycloudflare.com'] in next.config.ts.

Tripo3D API 403 errors The Tripo3D task endpoint returned consistent 403s due to API key permission limitations. The three-tier fallback architecture meant SF3D local generation kicked in automatically — no demo interruption.

Port conflicts The ML service on port 8000 was repeatedly killed by other processes. We moved it to 8765 (ML_PORT env var) and added a dedicated startup script.

Accomplishments that we're proud of

The SHAP waterfall chart This is the feature that makes SmartPick credibly different from a marketing page. Every return-risk score comes with a full feature-by-feature breakdown explaining why. Materials quality pushes the risk down. High price raises expectations (and risk). Shipping distance adds CO₂. We even included two intentionally irrelevant features in the training data (release year, product ID hash) — and SHAP correctly assigns them near-zero importance, demonstrating the model isn't overfitting to noise.

The three-tier 3D generation pipeline Tripo3D API → Meshy AI → SF3D local. The system tries cloud generation first for quality, falls back seamlessly to local GPU inference (~2s on RTX 5070) if APIs are unavailable, and caches all GLB files. Users get an interactive 3D model they can rotate and inspect — something you can't get from any Amazon listing.

Building the Chrome extension end-to-end Manifest V3, content script that auto-extracts product data from Amazon DOM (ASIN, title, price, brand, image, rating), badge injection next to the buy button, floating panel with eco data, and a popup that queues products for comparison — all communicating with the same Next.js backend.

Quantified environmental impact We didn't just say "sustainable." We computed: each comparison avoids ~1.2 kg of return-shipping CO₂. At 10,000 users making 5 comparisons each, that's 60,000 kg CO₂ avoided. Users see their personal tally on their profile. The impact is visible, specific, and earned.

Shipping the whole stack in a hackathon XGBoost ML service + SHAP explainability + three-tier 3D generation + Claude AI + Auth0 + Chrome extension + 8-tab comparison dashboard — all integrated and live.

What we learned

Explainability builds trust more than accuracy A 94% accurate model that shows no reasoning is less trustworthy to users than an 85% accurate model with a clear waterfall chart. SHAP transformed the ML score from a number into a story.

The information gap is the real enemy, not buyer regret Users don't return things because they're unhappy — they return things because the product didn't match what they imagined. Our preference wizard closes that imagination gap before the purchase.

Sustainable choice = easy choice We didn't ask users to be more eco-conscious. We built eco-impact data into the natural flow of a purchase decision. The CO₂ score shows up whether users care about it or not. That's the only way sustainability scales.

Fallback architecture is not optional Cloud APIs fail. Rate limits hit. Ports conflict. Every layer of the stack needed a fallback: 3D generation has three tiers, the ML service degrades gracefully, image URLs have CDN backups. Planning for failure made the demo reliable.

Auth0 version mismatches cost hours Never assume the installed package matches the docs you're reading. Check the actual installed version first — the v3/v4 migration cost us most of a session.

What's next for SmartPick

Immediate (post-hackathon)

Live Amazon data integration via Keepa API (real price history) and Perplexity AI (real-time product complaints from Reddit/forums)
Expand Chrome extension to Walmart, Best Buy, Target
Add more product categories: Home & Garden, Pet Supplies, Grocery

Near-term

Personalized return-risk model — learns from each user's own purchase and return history via Auth0 profile
Shareable comparison links so friends can decide together
Mobile app (React Native) with visual search — point camera at product for instant SmartPick analysis
AR try-on using the 3D models (headphones placed on your face via camera)

B2B

REST API for retailers to embed SmartPick return-risk scores directly in their product pages
Aggregate anonymized return data → seller insights: "Your product has 34% return risk because of poor packaging recyclability — here's how to fix it"
Amazon seller dashboard integration

Scale

Partner with carbon offset programs so unavoidable returns are automatically offset
Expand ML training to real Amazon return data (if partnership possible)
"What if Amazon showed you this before checkout?" — pitch to Amazon for native integration