TryIt AI — Universal eCommerce VTO & Skin AI

Universal Chrome extension & agentic fitting room powered by YouCam S2S APIs and Gemini. Cross-chains try-ons across stores and validates skin compatibility before checkout

Comment

Inspiration

Online fashion and beauty shopping has a persistent flaw: guesswork.

Shoppers face uncertain sizing charts, clashing color undertones, and severe return rates that average over 30% across the apparel industry. In beauty and skincare, consumers regularly experience adverse reactions from purchasing active chemical ingredients (like high-percentage AHAs or retinoids) that directly conflict with their delicate skin barrier.

Existing virtual try-on tools are siloed inside single walled-garden apps and rarely support multi-item outfit layering or cross-store shopping. As a software and Flutter developer, I wanted to build TryIt AI to bridge this gap: an open, universal browser extension and storefront orchestrator powered by Perfect Corp's YouCam S2S REST APIs and Google Gemini 2.5 Flash that injects a photorealistic virtual fitting room and clinical dermal barrier analyst directly beside any Add to Cart button across the web.

What it does

TryIt AI operates in two complementary modes:

  • Universal Chrome Extension (Manifest V3): Automatically detects product pages on any eCommerce storefront (including live stores like ZARR, Edenrobe, Zara, Sephora, Amazon, and Shopify storefronts) and injects a sleek "✨ TryIt with AI" button directly beside the native Add to Cart button.

  • Interactive Demo Store Sandbox (GET /demo-store): A dual-access web storefront allowing hackathon judges to immediately test virtual try-ons without installing any extension.

Core Capabilities

  • Multi-Pipeline Routing: Automatically routes scraped products to the right YouCam S2S v2.0 pipeline across apparel (cloth-v4), footwear (shoes-transfer), luxury bags (bag), accessories (scarf and hat), and cosmetics (makeup-vto, skin-analysis).

  • Cross-Store Outfit Chaining (POST /api/tryon/chain): Allows shoppers to pin a garment from Store A (e.g., an emerald trench blazer from ZARR or Zara) and chain it with items from Store B (e.g., Chelsea boots from Nordstrom), feeding the rendered output image of the first task directly into the second for a unified head-to-toe render.

  • Dermal Barrier & Skincare Telemetry: Evaluates 6 quantitative skin biomarkers (Moisture, Redness, Acne, Pores, Wrinkles, Texture) from YouCam Skin Analysis against cosmetic active ingredients to warn shoppers of potential irritation before checkout and recommend soothing alternatives.

  • 8 Diverse Shopper Personas & Custom Profiles: Features male and female studio models across Asian, African, American, and Latin backgrounds, plus support for uploading custom shopper photos and saving generated looks into personal profile galleries.

  • Natural-Language Styling Edits (POST /api/tryon/edit-command): Shoppers can type conversational commands like "Add obsidian leather boots" or "Remove last item" to dynamically manipulate their virtual look on the fly.

How we built it

  • Chrome Extension (Manifest V3): Built with lightweight vanilla JavaScript, utilizing background service workers to handle CORS-free communication and content scripts that extract Schema.org JSON-LD, OpenGraph tags, and DOM product data.

  • Backend Architecture: Developed with Python 3.11/3.13 and FastAPI, containerized with Docker, and deployed on Google Cloud Run for zero-timeout asynchronous task execution and global low latency.

  • YouCam API Client (backend/youcam_client.py): An asynchronous client orchestrating YouCam S2S v2.0 REST endpoints with split-polling architecture and built-in normalization for endpoint-specific parameter quirks.

  • Agent Orchestrator (backend/agent_orchestrator.py): Integrates Google Gemini 2.5 Flash as an autonomous router, stylist, and cosmetic chemist to classify items in under 1.5 seconds and synthesize pre-checkout verdicts.

Challenges we ran into

  • YouCam S2S Endpoint Parameter Quirks: Each YouCam API endpoint has strict schema nuances (for example, the bag endpoint rejects the style parameter with HTTP 400, while scarf and hat require an explicit style: "random" parameter for natural draping). I built a resilient translation layer in my client that automatically normalizes parameters per endpoint.

  • Cross-Store Image Chaining: Passing outputs across asynchronous tasks required piping intermediate high-resolution output URLs as new source inputs without compounding compression artifacts or losing posture alignment. I solved this by architecting an asynchronous task pipeline (POST /api/tryon/chain) that preserves high-fidelity output URLs and passes calibrated perspective parameters (fitting_mode: "auto_perspective").

  • Web Server Threading & Hosting Migration: I initially tested deployment on shared cPanel hosting via Phusion Passenger WSGI, but encountered thread-deadlocks with a2wsgi background event loops during multi-task YouCam polling. I resolved this by containerizing the FastAPI service with Docker and migrating it to Google Cloud Run, achieving zero-timeout async task polling and global CDN delivery.

Accomplishments that we're proud of

  • Seamless DOM Injection: The Chrome Extension cleanly injects into live eCommerce storefronts like zarr.com.pk and edenrobe.com with zero layout breakage or styling conflicts.

  • Comprehensive Automated Test Suite: Built an automated pytest suite (test_app.py) verifying all YouCam endpoints, split-polling logic, multi-store chaining, dermal conflict detection, and text command edits.

  • Dual-Mode Testing: Anyone can experience the full TryIt AI agent either through the unpacked Chrome Extension or directly through the hosted /demo-store web interface without installing anything.

What we learned

  • Deep practical mastery of Perfect Corp's YouCam S2S REST APIs across apparel, footwear, accessories, makeup, and clinical skin analysis.

  • Structuring multi-modal AI pipelines that combine visual biometric telemetry with LLM chemical reasoning to produce actionable pre-purchase advice.

  • Architectural patterns for low-latency asynchronous task polling in consumer-facing browser extensions and cloud backends.

What's next for TryIt AI — Universal eCommerce VTO & Skin AI

  • Cross-Platform Flutter Mobile App (Android & iOS): As an experienced Flutter developer, my immediate next step is building the official TryIt AI Mobile App for Android and iOS using Flutter. It will feature:

    • An in-app shopping browser that injects the TryIt AI fitting room onto mobile eCommerce sites.
    • Native camera integration for instant real-time face and full-body scans.
    • Personal lookbook gallery synchronized with the cloud backend.

  • 1-Click Multi-Store Cart Checkout: Enabling shoppers to purchase their chained multi-store outfits through a single consolidated checkout.

  • Real-Time Video Stream Try-On: Integrating YouCam AR Live SDKs for real-time webcam and mobile camera motion tracking.

..............

Built With

Share this project:

Updates

Submission history