VeraFitAI

Inspiration

Over 30% of online apparel purchases are returned, resulting in billions in reverse-logistics costs, massive restocking overhead, and severe environmental waste. The root cause is simple: every returned garment is a failed prediction made at the point of purchase.

Most virtual try-on (VTO) tools operate as simple visual "gimmicks"—rendering a single image without evaluating drape stability, fabric safety, or true color compatibility with a user’s unique skin biometrics. We asked ourselves: What if e-commerce platforms could mathematically and clinically stress-test purchase certainty before checkout?

VeraFit AI was born to turn returns from a post-purchase cost center into an explainable, pre-purchase certainty signal.

What it does

VeraFit AI is a multi-agent Purchase Certainty Engine that evaluates fit, seasonal color harmony, and fabric-to-skin safety on a personalized digital mannequin before the shopper completes a purchase.

Core Capabilities:

1. Fit Stability Stress-Testing: Executes 3 parallel VTO renders on the shopper’s photo using YouCam’s AI cloth try-on engine and calculates Structural Similarity Index Measure (SSIM) variance across render pairs. Unstable drapes trigger fit warnings and visual difference heatmaps.
2. Perceptual Color True-Matching: Extracts garment colors and calculates the perceptual color difference ($\Delta E$ in CIELab color space) against the shopper’s calibrated seasonal palette (Cool Winter, Warm Autumn, etc.) and skin tone.
3. Fabric & Allergen Safety Audit: Cross-references the garment’s material composition map against shopper allergies (e.g., wool, nickel, synthetics) and dermal conditions (rosacea, eczema, friction sensitivity), applying deterministic safety multipliers.
4. AI Plain-Language Verdicts: Fuses multi-agent signals into a weighted Keep-Probability Score and renders an actionable verdict (STRONG_BUY, CONSIDER_CAUTION, HIGH_RETURN_RISK) with full diagnostic explainability.
5. Continuous Learning Loop: Shoppers log keep/return/purchase outcomes, allowing preference bias vectors to continuously self-calibrate.
6. B2B Vendor-Isolated Merchant Intelligence: Provides brand merchants with real-time fleet analytics, return-risk SKU early warnings, supplier defect tracking, and inventory clearance recommendations.

How we built it

VeraFit AI is built with a decoupled modern architecture orchestrating specialized AI agents, computer vision, and deterministic math kernels:

• Multi-Agent Orchestration (LangGraph): Uses a parallel fan-out / fan-in topology where 5 specialized domain agents (VTO Agent, Color Agent, Fabric Agent, Personalization Agent, and Purchase-History Agent) run concurrently and feed directly into a Synthesis Agent.
• Computer Vision & Virtual Try-On: Powered by YouCam AI Skin Analysis & Facial Color Tones API (/s2s/v2.1/task/skin-analysis, /s2s/v2.0/task/skin-tone-analysis) and YouCam Clothes VTO (cloth-v4), backed by offline mock fallbacks.
• Deterministic Math Kernels: Built with NumPy, OpenCV, and scikit-image to compute pixel-level SSIM variance heatmaps and CIELab $\Delta E$ perceptual color vectors.
• Backend API & Persistence: Built using Python 3.14, FastAPI, SQLAlchemy (async), and aiosqlite / SQLite, with LLM routing managed via LiteLLM (supporting GPT-4o, Anthropic, Gemini, Groq, or local Ollama).
• Frontend Experience: Built with React, TypeScript, Vite, and Zustand, featuring interactive VTO sliders, an AI X-Ray explainability inspector, and role-based persona switching.
• Deployment & Packaging: Unified multi-stage Docker container ready for single-container execution or cloud deployment to AWS ECS Fargate / EC2.

Challenges we ran into

1. Quantifying "Fit Uncertainty": A single generative VTO image can hide fitting flaws or artifact hallucinations. We solved this by generating multiple asynchronous render passes and running pairwise SSIM matrix calculations to detect drape instability algorithmically.
2. Balancing LLM Reasoning with Deterministic Safety: LLMs can hallucinate safety ratings. We enforced strict clinical guardrails: allergen conflicts apply a hard deterministic 0.40 penalty multiplier to the final score before the Synthesis LLM generates the plain-language explanation.
3. Low-Latency Agent Concurrency: Orchestrating 6 separate agents in sequence would make checkout flow sluggish. We designed the LangGraph workflow so all domain agents evaluate the garment concurrently in parallel, converging into the Synthesis Agent in a single pass.
4. Perceptual Color Science: Standard RGB Euclidean distance does not accurately reflect human perception of skin-to-garment contrast. We converted all color calculations into perceptual CIELab space and mapped them to 12-season color analysis standards.

Accomplishments that we're proud of

• True Pre-Purchase Explainability: Moving beyond black-box scores with our AI X-Ray Inspector, which visually decomposes verdicts into SSIM heatmaps, $\Delta E$ color swatches, and allergen warnings.
• End-to-End Multi-Agent Architecture: Implementing a fully functional parallel LangGraph state machine where deterministic computer vision, rule engines, and LLMs work in harmony.
• Dual B2C & B2B Value Proposition: Empowering shoppers to avoid regret purchases while simultaneously equipping merchants with vendor-scoped analytics to flag return-prone SKUs before committing inventory.
• Full-Body Persona Fidelity: Gender-aware try-on pipelines that seamlessly tailor full-body mannequin imagery, men's/women's catalogs, and biometric sensitivity profiles.

What we learned

• Deterministic Math + Generative AI is the sweet spot: Pure generative models struggle with consistent accuracy, while pure math lacks personalized articulation. Combining deterministic computer vision (SSIM, CIELab) with generative synthesis produces both trust and readability.
• Returns are multifaceted: Fit alone is not the sole reason for returns—fabric allergies, tactile friction, and color-undertone clashes are major hidden drivers that traditional sizing charts completely ignore.
• Multi-agent state machines simplify complexity: Structuring domain concerns into isolated, testable agents made it easy to tune weights and maintain clear API boundaries.

What's next for VeraFitAI

Following our shipped core engine (M1–M3), our roadmap progresses across three clear horizons:

• M4 — Hardening & Production Scale (Near-Term): Transitioning from demo personas to enterprise JWT authentication, migrating SQLite to managed PostgreSQL with S3 object storage for high-res VTO renders, and integrating worker task queues for heavy CV/VTO jobs.
• M5 — Deeper Prediction Engine:
  • On-Device Biometric Calibration: Mobile camera & flash auto-detection of seasonal color palettes and dermal concerns.
  • Generative 3D Fit Simulation: Moving beyond 2D SSIM to multi-scan 3D volumetric avatars with dynamic fabric physics and drape stress-testing.
  • Clinical Fabric Knowledge Graph: Expanding the fiber ↔ allergy matrix to include certified eco/dermatological standards and category-level return drift.
• M6 — Enterprise Network Effects & Checkout Integration:
  • E-Commerce & Whitelabel Plugins: Embedding pre-purchase certainty verdicts directly into Shopify, Salesforce Commerce Cloud, and WooCommerce product pages.
  • Return-Insurance Risk Scoring (B2B): Partnering with checkout & logistics providers to dynamically price return insurance and offer personalized zero-return discounts.
  • Cross-Brand Fleet Intelligence: Aggregating anonymized SKU return signals across merchants to help buying teams avoid problematic garment cuts prior to manufacturing

Built With

• amazon-web-services
• artificial-intelligence
• b2b-saas
• cielab-color-matching
• computer-vision
• docker
• ecommerce
• explanable-ai
• fashion-tech
• fastapi
• generative-ai
• image-processing
• langgraph
• llm
• multi-agent-systems
• python
• react
• retail-analytics
• returns-reduction
• skin-analysis
• ssim
• typescript
• virtual-try-on
• vite
• youcam-api