Cartify - Hackathon Submission

Inspiration

Online shopping has become overwhelming. Users spend hours scrolling through thousands of products, applying filters, and comparing prices. We asked: What if shopping could be as simple as having a conversation?

Inspired by how we naturally ask friends for recommendations ("I want to cook biryani, what do I need?"), we built Cartify - an AI shopping assistant that understands intent, mood, and budget to find exactly what you need.

What it does

Cartify transforms e-commerce into a conversational experience:

Natural Language Shopping: Instead of searching "rice" + "chicken" + "spices" separately, just say "I want to cook biryani under $50" and Cartify finds all the essential ingredients within your budget.
Voice-Enabled Shopping: Speak your requests hands-free using ElevenLabs speech-to-text. Perfect for busy users or accessibility needs.
Smart Recommendations: The AI understands context - "birthday party supplies" returns decorations, cake, and balloons, not random party-related items.
One-Click Cart: Add all AI-suggested products to your cart instantly.
Budget Optimization: Cartify respects your budget and finds the best products within your price range.

How we built it

Frontend:

React 18 with TypeScript for type-safe development
Tailwind CSS for responsive, modern UI
Zustand for lightweight state management
Vite for fast development and optimized builds

Backend (Raindrop Platform on Vultr):

LiquidMetal AI's Raindrop Platform for serverless API deployment
Vultr cloud infrastructure for reliable AI workloads
Three main endpoints:
- /api/cartify-agent - Processes natural language queries using Groq AI (Llama 3.1)
- /api/speech-to-text - Voice transcription via ElevenLabs
- /api/products - Real-time product search via SerpAPI (Walmart)

AI Pipeline:

User query → Groq AI extracts intent, mission, and required items
For each item → SerpAPI fetches best matching Walmart products
Budget filter → Returns optimized product list
Frontend displays → User adds to cart

Database:

Supabase for user authentication, cart persistence, and order history

Challenges we ran into

CORS Issues: SerpAPI doesn't allow browser requests. We solved this by routing all API calls through our Raindrop backend.
Too Many Products: Initial AI responses returned 50+ products for simple queries. We refined the Groq prompt to return only 5-8 essential items and limited SerpAPI results to 1 best match per ingredient.
Windows Compatibility: Raindrop CLI had issues with npx spawn on Windows. We patched the bundle files to use shell: true for cross-platform compatibility.
Voice Transcription Latency: ElevenLabs API needed proper audio format handling. We implemented WebM recording with proper blob handling for reliable transcription.

Accomplishments that we're proud of

Seamless AI Integration: The conversation feels natural - users don't need to learn special commands or syntax.
Smart Context Understanding: Saying "cook biryani" returns rice, chicken, and spices - not biryani-flavored chips or cookbooks.
Production-Ready Deployment: Fully deployed on Raindrop/Vultr with proper error handling, CORS configuration, and environment variable management.
Voice Shopping Works: Real-time speech-to-text that actually understands shopping queries accurately.
Clean, Responsive UI: Works beautifully on desktop and mobile with a modern indigo theme.

What we learned

Prompt Engineering Matters: The difference between 50 random products and 7 perfect ingredients came down to how we structured the Groq AI prompt.
Serverless Simplifies Deployment: Raindrop Platform made deploying AI workloads incredibly simple - no server management, just code and deploy.
API Design for AI: Building APIs that serve AI responses requires thinking about response structure, error handling, and graceful degradation.
Voice UX is Tricky: Voice interfaces need clear feedback states (listening, transcribing, processing) to feel responsive.

What's next for Cartify

Multi-Store Support: Expand beyond Walmart to Amazon, Target, and local grocery stores for price comparison.
Personalized Recommendations: Learn user preferences over time - dietary restrictions, favorite brands, past purchases.
Recipe Integration: Connect with recipe APIs so users can say "make Gordon Ramsay's beef wellington" and get the exact ingredients.
Voice Response: Add ElevenLabs text-to-speech so Cartify can speak back recommendations.
Shopping Lists: Save and share AI-generated shopping lists with family members.
Price Alerts: Notify users when items on their wishlist drop in price.
Mobile App: Native iOS/Android apps with barcode scanning and in-store navigation.