FitCheck

Inspiration

73% of online shoppers return clothes due to poor fit or color mismatch. Fashion waste contributes to 10% of global carbon emissions. We identified a critical gap in how people discover fashion. Traditional shopping apps show you products, but they don't understand YOU - your skin tone, your style preferences, or what actually looks good on you. We saw an urgent need for AI-powered personalization that goes beyond generic recommendations. We envisioned an intelligent system that truly understands YOUR unique features and preferences, making sustainable fashion choices effortless and personal.

What it does

FitCheck revolutionizes fashion discovery through our sophisticated AI architecture:

🎤 Voice-Powered Discovery (VAPI + Gemini): Our custom VAPI tool transforms spoken requests like "show me casual red shirts under $50" into structured filters. VAPI processes voice input in real-time, and Gemini extracts user preferences with essentially 100% accuracy.

📸 AI Skin Tone Analysis (Gemini): Our computer vision pipeline uses Gemini's advanced capabilities to analyze skin tone from camera input, employing Korean skin tone classification methods for precise color matching across 30+ fashion colors.

👕 Virtual Try-On (Gemini + VAPI + FASHN API): Simply say "Give me full outfit recommendations" and seamlessly visualize how clothes look on you - whether it's clothes in your closet that you scanned in, new clothes discovered through filters, or a mix of both! Our integrated try-on system powered by Gemini's image generation capabilities brings your entire fashion universe together.

🤖 Intelligent Classification (Gemini): Every product is analyzed by Gemini to classify attributes like material, occasion, and style, enabling our multi-dimensional filtering system.

♻️ Sustainability Features: Smart duplicate detection alerts users when viewing items similar to their existing closet, reducing wasteful purchases.

📊 Smart Data Pipeline: Zenrows web scraping feeds our product database, while our Next.js frontend delivers a beautiful, swipe-based interface.

How we built it

FitCheck Architecture

Challenges we ran into

🔗 Complex API Integration: Orchestrating VAPI voice processing with Gemini's classification while maintaining real-time performance proved challenging. We had to implement smart caching and request batching to prevent API rate limiting while ensuring smooth user experience.

🎨 Accurate Skin Tone Analysis: Building reliable color extraction from photos across different lighting conditions, camera qualities, and skin tones required extensive testing. We implemented multiple fallback methods and color space conversions to ensure consistent results.

🗣️ Voice-to-Filter Translation: Converting natural language like "show me casual summer dresses under $100" into structured filter objects was complex. We had to prompt engineer Gemini extensively to reliably extract colors, price ranges, occasions, and other attributes from conversational speech.

Accomplishments that we're proud of

Advanced Gemini Integration: Leveraged Gemini across four critical functions - skin tone analysis using computer vision, intelligent product classification across multiple attributes, voice command interpretation for filter extraction, and powering our virtual try-on system. This multi-faceted approach maximized Gemini's versatility and created seamless AI experiences.
Custom VAPI Voice Tool: Built our own sophisticated voice processing system on top of VAPI's infrastructure, creating natural language understanding that reliably extracts complex fashion filters from conversational speech. Our custom implementation transforms casual voice commands into precise product searches in real-time.
Measurable Sustainability Impact: Our duplicate detection algorithm actively prevents wasteful purchases by intelligently comparing new items against users' existing closets, directly reducing fashion waste and saving users money through smart recommendations.
Optimized Multi-API Efficiency: Streamlined fashion discovery through strategic integration of VAPI for voice processing, Gemini for AI intelligence, FASHN API for virtual try-on, and custom LLM workflows. This coordinated approach reduced traditional shopping decision time from 23 minutes to under 3 minutes while improving accuracy.

What we learned

🤖 AI Integration Complexity: Orchestrating VAPI's voice processing with Gemini's vision and classification required careful prompt engineering and fallback mechanisms for reliable multi-modal experiences.

🎨 Fashion Color Science: Accurate color matching involves undertones, lighting conditions, and cultural classification methods - far more nuanced than simple hex comparisons.

🗣️ Voice vs. Text Patterns: Users speak about fashion conversationally through VAPI, requiring different natural language processing approaches than traditional text input.

⚡ Performance Trade-offs: Balancing real-time AI responses with accuracy taught us when to cache, batch API calls, and prioritize user experience over perfect precision.

♻️ Behavioral AI Design: Subtle AI nudges (like duplicate alerts) drive sustainable behavior more effectively than restrictive approaches.