DermAtelier

Inspiration

Online beauty and apparel retail faces an USD 816B annual reverse-logistics crisis, with 44.2% return rates in shade-sensitive categories and 62%+ landfill write-offs on opened cosmetics that cannot be restocked. When we investigated why shoppers return items they already previewed online, we uncovered a fundamental blind spot:

Standard Virtual Try-On treats living human skin like flat Photoshop paper.

• It renders a high-silica Ultra-Matte Blue-Crimson lipstick onto a portrait without knowing the shopper's stratum corneum hydration raw score is 55.8 (guaranteed perioral micro-flaking within 4 hours) or that bilateral malar redness (54.2) will be visually magnified +21% by cool crimson simultaneous contrast.
• It previews opaque titanium-heavy foundations without flagging grey-violet ash cast on Fitzpatrick VI deep warm-golden skin, or acidic chartreuse knitwear that washes out olive and cool-rosy complexions.
• Worst of all, shoppers add incompatible clinical actives (1.0% Pure Retinol + pH 3.4 10% Glycolic Acid) into the same bag, stripping a reactive skin barrier and triggering product returns.

We built DermAtelier (dermatelier.vercel.app) so Perfect Corp.'s clinical Skin AI actively governs Virtual Try-On, 4-Hour Barrier Wear Forecasting, and Checkout.

What it does

DermAtelier is a closed-loop luxury clinical storefront and B2B2C Return-Shield platform chaining three live-verified Perfect Corp. YouCam S2S v2.0 APIs:

1. Stage 01 — 14-Concern Clinical Skin Telemetry + CIE L*a*b* ITA° (/s2s/v2.0/task/skin-analysis)
   • Scans a live webcam capture, uploaded portrait, or three unretouched Fitzpatrick II / IV / VI studio twins across 14 standard dst_actions (wrinkle, pore, texture, acne, redness, oiliness, moisture, age_spot, radiance, dark_circle_v2, firmness, eye_bag, droopy_upper_eyelid, tear_trough).
   • Computes objective dermatological Individual Typology Angle in CIE L*a*b* color space alongside Perfect Corp's dual consumer ui_score and unmodified clinical raw_score, Barrier Integrity (/100), Deepgram Aura-2 Neural Audio Briefings, and remapped Spatial Zone Alpha Masks with a 2.2x Clinical Loupe:

$$\text{ITA}^\circ = \arctan\left(\frac{L^* - 50}{b^*}\right) \times \frac{180}{\pi}$$

1. Stage 02 — Dermal-Governed Split-Slider VTO + T+4h Wear Forecast (/s2s/v2.0/task/makeup-vto & /s2s/v2.0/task/cloth)

   • Evaluates every cosmetic and upper-body garment SKU against the shopper's live Skin AI telemetry (moisture x finish demand, pore/acne x occlusion risk, redness x simultaneous color contrast, melaninIndex x Fitzpatrick I–VI shade maps).
   • Intercepts high-return clashes before checkout, auto-selects a Dermal-Calibrated Counterpart, and renders an accessible WAI-ARIA Split-Slider VTO Comparator—complete with a T+0h Fresh vs. T+4h Barrier Wear toggle simulating 4-hour matte micro-creasing, pore ostia pooling, and +0.5 shade sebum oxidation vs. ceramide-satin stability.

2. Stage 03 — Biochemical Cart Guardrail, Telemetry Passport & Live Shopify PDP Simulator

   • Audits the shopping bag for active pH collisions (1.0% Retinol x pH 3.4 Glycolic/Salicylic Acid) and VTO undertone clashes, offering 1-Click Bag Calibration, an auto-sequenced Wear Now · AM Protect · PM Renew regimen, and a downloadable Dermal Telemetry Passport (.json) bound to the order.
   • Provides eCommerce VPs with a Merchant Return-Shield Console featuring a dynamically computed SKU x Fitzpatrick II–VI Telemetry Matrix, interactive unit-economics sliders (USD 238,700/mo preserved margin at 25k monthly orders), and a Live Interactive Shopify PDP Buy-Box Simulator.

How we built it

• Full-Stack Architecture: Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS v4, sharp, fflate, Deepgram Aura-2 Neural TTS (/api/youcam/tts), and Vitest, deployed on Vercel.
• Dual-Framing Face-ROI & Elliptical VTO Compositing (src/lib/youcam/image-framing.ts): Automatically extracts a 768x906 macro face ROI (~70% face width) to satisfy /task/skin-analysis and /task/makeup-vto, while preserving full 896x1200 upper-torso framing for /task/cloth—inverse-projecting returned PNG alpha masks and cosine-feathered VTO renders seamlessly back onto the full portrait.
• Live-Verified YouCam S2S v2.0 Gateway (src/lib/youcam/client.ts): Implements /s2s/v2.0/file/{service}, presigned AWS S3 PUT with SigV4 header deduplication, async task execution across skin-analysis, makeup-vto (foundation, lip_color, blush), and cloth, adaptive polling, and dual JSON/ZIP (fflate) parsing.
• SHA-256 Unit-Shield Vault (src/lib/youcam/cache.ts): Content-addresses every request by (service, sha256(imageBytes), canonicalParams) and persists 2-hour expiring S3 URLs locally so repeat runs consume 0 API units.

Challenges we ran into

1. Solving the Dual-Framing Paradox Across Skin AI vs. Clothes VTO: During live integration against yce-api-01.makeupar.com, we discovered /s2s/v2.0/task/skin-analysis strictly requires face width >60% of frame (error_src_face_too_small), whereas /s2s/v2.0/task/cloth requires full upper-torso framing (~38% face width). Rather than forcing shoppers to upload two separate photos, we built a sharp Dual-Framing Pipeline that crops a macro face ROI for Skin AI + Makeup VTO and mathematically inverse-maps returned masks and elliptical-feathered renders onto the 896x1200 Clothes VTO portrait.
2. Strict OpenAPI Category Schemas & Expiring S3 Presigned URLs: Each makeup-vto category enforces distinct payload invariants (coverageIntensity/glowIntensity without texture on foundation, shape + style on lip_color, pattern: { name: "2colors6" } on blush), and returned S3 masks expire after 7200s. We live-verified every category against the production cluster and built immutable local asset persistence.
3. Objective Dermatological Colorimetry: Instead of naive RGB luminance averages, we implemented sRGB gamma linearization to CIE XYZ (D65) to CIE L*a*b* and $\text{ITA}^\circ$ computation to drive inclusive Fitzpatrick I–VI shade calibration.

Accomplishments that we're proud of

• Live-verifying all three Perfect Corp. YouCam S2S v2.0 APIs (skin-analysis, makeup-vto, cloth) in one unified consumer + B2B merchant application while preserving 987 / 1,000 hackathon API units via SHA-256 Unit-Shield.
• Pioneering T+4h Barrier Wear Forecasting—bridging static second-zero VTO with dynamic 4-hour dermatological wear physics.
• Passing 16/16 automated Vitest unit and Next.js route-handler tests covering CIE L*a*b* $\text{ITA}^\circ$ colorimetry, catalog referential integrity, S2S v2.0 schemas, and zero-unit cache hits.

What we learned

• Clinical skin telemetry becomes exponentially more valuable when treated not as an isolated quiz result, but as a real-time governance layer over generative VTO parameters, shade selection, and cart formulation safety.
• Careful server-side image geometry normalization (sharp) and content-addressed caching turn multi-second S2S computer vision pipelines into sub-20ms interactive shopping experiences.

What's next for DermAtelier

• Packaging the 1-line Return-Shield <script> tag as a native Shopify Theme App Extension with webhook-backed return-rate attribution.
• Longitudinal multi-week barrier tracking so skincare replenishments automatically titrate retinoid and acid concentrations as raw_score hydration and erythema telemetry improve.

Built With

• computer-vision
• deepgram
• docker
• next.js
• node.js
• perfect-corp
• react
• sharp
• tailwindcss
• typescript
• vercel
• vitest
• youcam-api