Inspiration

Skincare shopping today is an overwhelming maze. Consumers are flooded by thousands of cosmetic formulations, overlapping chemical actives, conflicting influencer advice, and rapidly compounding costs. A beginner standing in front of a mirror or an eCommerce store rarely knows which active ingredients are appropriate for their visible skin texture, nor whether their morning routine will inadvertently trigger barrier sensitivity by clashing active ingredients like Retinol and Salicylic Acid.

We asked ourselves: What if shopping for skincare was as intuitive, precise, and transparent as consulting an expert aesthetician?

We built SKINMATE AI to bridge clinical facial computer vision and real-world eCommerce, helping users understand their skin at the exact moment of decision, discover verified formulations tailored to their unique skin profile, and build complete AM/PM routines strictly within their financial budget.


What it does

SKINMATE AI is an intelligent personal skincare intelligence and smart shopping platform built from the ground up:

  • 🔬 16-Metric Clinical Facial Diagnostics: Powered by the Perfect Corp YouCam Skin AI REST API, SKINMATE AI analyzes 16 visible cosmetic characteristics from a single front-facing selfie or live webcam scan—including pores, texture, moisture, oiliness, redness, wrinkles, dark circles, eye bags, and radiance.
  • 🗺️ Canvas Mask & Heatmap Overlays: Projects YouCam's visual detection masks directly over the user's face, pinpointing exact regions where pore dilation, texture irregularity, or oil clusters are concentrated.
  • 🛡️ Smart Active Conflict & Safety Guard: A deterministic contraindication matrix analyzes active cosmetic ingredients (AHA/BHA, Retinoids, Vitamin C, Niacinamide, Benzoyl Peroxide), preventing hazardous combinations and enforcing mandatory daytime SPF.
  • 💰 Combinatorial Budget Optimizer: Solves an optimal 3-step morning and evening regimen strictly within user-defined financial ceilings (e.g., ₹500, ₹1,200, ₹1,500, or ₹3,000) with real-time price differential calculation.
  • 🤖 Agentic Conversational Concierge: A Groq-powered multi-turn AI shopping assistant equipped with function calling to search verified products, compute price differentials, answer skincare questions, and suggest one-click routine updates.
  • 🌓 Seamless Responsive UI & Dual Theme: Features a luxury aesthetic with fluid dark/light modes, animated sun/moon theme toggle, interactive concern radar breakdown, and auto-capture camera guidance.

How we built it

SKINMATE AI was engineered using a high-performance, modern full-stack architecture:

  1. Perfect Corp YouCam Skin AI REST API Integration:

    • Uses YouCam's asynchronous S2S API pipeline: pre-signed image upload (/s2s/v2.0/file), task creation (/s2s/v2.0/task/skin-analysis), and polling (/s2s/v2.0/task/skin-analysis/{task_id}).
    • Extracts quantitative ui_score and raw_score across all 16 dermatological concerns.
    • Downloads and renders YouCam detection masks with toggleable visual overlays.
    • Built-in client retry, error recovery, and dual-mode fallback ensuring 100% demo uptime.
  2. Frontend & Design System:

    • Built with Next.js 15 (App Router), React 19, and TypeScript 5.7.
    • Styled with Tailwind CSS 3.4 and Framer Motion 11 for fluid micro-interactions and smooth page transitions.
    • Comprehensive dark/light theme support with high-contrast obsidian slate and botanical emerald design tokens.
  3. Agentic Conversational Engine:

    • Powered by Groq (openai/gpt-oss-120b) for sub-second agentic shopping reasoning.
    • Implements custom tool-calling agents to query product catalogs, verify INCI ingredient lists, and recommend budget-optimized swaps.
  4. Data & Persistence Layer:

    • Integrated with Supabase for cloud synchronization and client-side session state for instant local testing.

Challenges we ran into

  • Asynchronous Polling & Mask Alignment: Synchronizing YouCam's asynchronous task lifecycle with client-side camera captures required building a robust polling pipeline with exponential backoff and timeout handling, as well as mapping bounding boxes and overlay masks cleanly onto responsive canvas elements.
  • Deterministic Active Clash Matrix: Developing an algorithmic ingredient clash detector that accurately maps complex INCI nomenclature and prevents negative active pairings (e.g., simultaneous exfoliants and retinoids) without false positives.
  • Combinatorial Budget Optimization: Calculating optimal product combinations across cleanser, treatment, and moisturizer tiers within hard budget constraints while prioritizing active ingredients targeting the user's lowest YouCam skin scores.

Accomplishments that we're proud of

  • End-to-End Working Experience: Built a fully operational, production-grade application featuring real-time camera scanning, 16-metric YouCam AI analysis, visual heatmap overlays, dynamic routine builders, and an AI conversational concierge.
  • Sub-Second Groq Agent: Seamlessly connected the skincare diagnostics with an agentic shopping assistant that can reason over product formulations and prices in milliseconds.
  • Flawless UI & Aesthetics: Delivered a luxury, responsive cosmetic brand experience in both dark and light modes with custom iconography, radar charts, and interactive product cards.

What we learned

  • Deep mastery of Perfect Corp's YouCam Skin AI API suite, understanding how computer vision models quantify subtle facial texture, hydration markers, and pigmentation clusters.
  • Best practices in building safe, consumer-friendly beauty eCommerce tools that demystify chemical actives while respecting cosmetic safety boundaries.
  • Structuring multi-modal agent workflows where computer vision outputs directly inform LLM decision-making and eCommerce catalog search.

What's next for SKINMATE AI

  • Expanding Virtual Try-On (VTO): Integrating YouCam's Apparel and Makeup VTO APIs to allow users to visualize tinted sunscreens, lip treatments, and cosmetics directly on their live video feed.
  • Longitudinal Skin Progress Tracking: Adding time-series comparison tools to measure visible skin improvements over 30, 60, and 90-day product cycles.
  • Dermatologist Tele-Consultation Bridge: Enabling users to export their structured YouCam diagnostic logs and ingredient histories to board-certified dermatologists for clinical review.
  • E-Commerce Direct Checkout: Integrating live shopping feeds and multi-retailer inventory sync for seamless one-click purchasing.

Built With

  • computer-vision
  • ecommerce
  • framer-motion
  • groq
  • llm
  • nextjs
  • react
  • skincare
  • tailwind-css
  • typescript
  • youcam-skin-ai
Share this project:

Updates

Submission history