BeautyLoop

Autonomous dermatological AI & dual AR try-on studio powered by YouCam APIs. Solves eCommerce return rates with clinical skin diagnostics, CIELAB colorimetry, and auditable formulation receipts.

Comment

Inspiration

Online beauty and skincare commerce faces a compounding crisis: over 30% of cosmetic purchases are returned due to shade mismatches, while aggressive formulation marketing conceals whether active ingredients will compromise a consumer's skin barrier. The result is massive retail return waste, consumer dissatisfaction, and widespread epidermal irritation.

Traditional virtual try-ons treat augmented reality as an isolated visual filter rather than a clinical retail decision tool. We built BeautyLoop to bridge this gap—transforming virtual try-on from a superficial novelty into an autonomous dermatological decision engine. By coupling clinical diagnostic biometrics with CIELAB colorimetry and dual AR simulation, BeautyLoop provides shoppers with an evidence-based shopping workflow supported by auditable formulation receipts.

What it does

BeautyLoop is a consent-first, end-to-end beauty and skincare consultation studio that operates in four synchronized stages:

  1. Dual AR Virtual Try-On Studio (Foundation + Lipstick): Rather than testing products in isolation, BeautyLoop pairs complex foundation coverage (configurable depth, intensity, and skin glow) with multi-texture lipstick simulations (Matte, Glossy, Shimmer). Shoppers inspect results using an interactive, touch- and mouse-responsive sub-pixel before/after split slider.
  2. 10-Biomarker Clinical Skin Diagnostic Engine: Powered by YouCam AI, BeautyLoop analyzes 10 cosmetic dimensions: pores, texture, acne, wrinkles, moisture, oiliness, redness, radiance, firmness, and eye bags. These are synthesized into a Skin Vitality Index (0–100) calculated through weighted barrier equilibrium.
  3. Autonomous Agentic Derm Concierge: Translates biometric findings into custom 4-step AM Defense and 3-step PM Nocturnal Cellular Repair protocols. It integrates an automated Contraindication Conflict Matrix that actively screens formulations to prevent dangerous active clashes (e.g., Vitamin C clashing with Retinol or high-percentage BHAs).
  4. CIELAB Colorimetry & Shade Matching: Computes facial, lip, eye, and hair color, mapping coordinates into CIE $L^a^*b^$ space. Using the CIE76 $\Delta E$ Euclidean color distance metric:

##\Delta E^_{ab} = \sqrt{(L^_2 - L^_1)^2 + (a^_2 - a^_1)^2 + (b^_2 - b^*_1)^2}##

BeautyLoop automatically matches natural undertones (Warm, Cool, Neutral, Olive) to exact retailer foundation catalogs (e.g., Maybelline Fit Me).

  1. Source-Backed Decision Card: Connects catalog products to verified retailer receipts with price observation dates, clearly distinguishing confirmed benefits from unverified marketing claims. Shoppers export a portable, copyable Decision Card summarizing their choices and biometric verification receipts.

How we built it

BeautyLoop is built as a production-grade monorepo featuring a clean Server-to-Server (S2S) architecture:

  • Frontend: React 19, TypeScript, Vite, and TailwindCSS for high-performance rendering. Custom HTML5 Canvas and CSS clipping pipelines drive the interactive before/after split comparison slider.
  • Backend Gateway: Node.js and Express running TypeScript. Request and response payloads are strictly validated using Zod schemas to guarantee runtime type safety and protect downstream API keys.
  • YouCam Provider Integrations (Perfect Corp S2S REST APIs):
    • /s2s/v2.0/task/makeup-vto: Dispatches foundation and lip simulations in a single unified payload, cutting API credit consumption by 50% and dramatically reducing network latency.
    • /s2s/v2.0/task/skin-analysis: Evaluates 10 clinical dermatological dimensions from facial imagery.
    • /s2s/v2.0/task/skin-tone-analysis: Extracts multi-region colorimetry metrics and CIELAB color profiles.
  • Asset Pipeline: Direct, secure AWS S3 presigned upload pipeline ensures high-resolution imagery transfers seamlessly to Perfect Corp's processing clusters without permanent server storage.

Challenges we ran into

  • Single-Payload Dual AR Orchestration: Aligning both foundation coverage and multi-finish lip color into one synchronous /makeup-vto payload required reverse-mapping RGB colorspace values to YouCam’s native color palettes while calibrating opacity and skin-glow ratios.
  • Color Distance Accuracy: Standard RGB distance formulas fail to represent human color perception across different skin undertones. We solved this by projecting YouCam's extracted skin color values into CIELAB space and calculating CIE76 $\Delta E$ Euclidean vectors against manufacturer shade databases.
  • Ethical AI & Consent Constraints: Balancing rich biometric diagnostics with user privacy required designing an ephemeral asset lifecycle where photos exist only in memory during task execution, paired with explicit consent gates and honest service status indicators.

Accomplishments that we're proud of

  • Complete, Cohesive Product: We delivered an end-to-end deployed application (https://beautyloop.onrender.com) featuring responsive UI, instant provider verification, interactive AR comparison, and clinical safety matrices.
  • True Multi-API Orchestration: Seamlessly orchestrated three distinct YouCam REST endpoints within a single consumer decision journey.
  • The Clinical Decision Card: Developed an exportable receipt mechanism that shifts the beauty shopping paradigm from "marketing hype" to "verifiable facts."
  • Production Engineering: Zero mock data in production; fully working Server-to-Server pipelines with structured error boundaries and type-safe Zod validation.

What we learned

  • How to effectively harness Perfect Corp's Server-to-Server APIs to handle complex, multi-attribute AR simulations efficiently.
  • The mathematical precision of CIELAB colorimetry and Delta-E algorithms in automating cosmetic shade matching.
  • Dermatological active contraindication logic and the profound retail value of protecting consumer skin barriers during checkout.

What's next for BeautyLoop

  • Direct Retailer Checkout Integration: Partnering with beauty marketplaces (Sephora, Ulta, Nykaa) to inject the Clinical Decision Card directly into digital carts.
  • Environmental & Seasonal Adaptation: Integrating local UV index, humidity, and pollution APIs to dynamically modulate the AM Photoprotection and PM Hydration recommendations.
  • Expanded Product Shade Embeddings: Scaling the CIE76/CIEDE2000 shade matching database to over 5,000 global foundation, concealer, and lipstick formulations.

Built With

Share this project:

Updates

Submission history