ToneMatch

AI-powered skin tone analysis that finds your perfect foundation match and color season — snap a selfie, get instant results.

Comment

ToneMatch — AI Skin Tone Analysis & Foundation Matching

Elevator Pitch

Snap a selfie. Discover your color season. Find your perfect foundation. Try it on virtually.

Inspiration

Finding the right foundation shade is frustrating — 70% of people buy the wrong shade on their first try. Color season analysis (Spring, Summer, Autumn, Winter) has been used by stylists for decades but remains inaccessible to most. We wanted to democratize professional-grade skin analysis using AI, making it instant and available to anyone with a phone.

What It Does

ToneMatch analyzes a selfie through Perfect Corp's AI skin tone API to:

  1. Detect your exact skin tone as a hex color value
  2. Classify your undertone (warm, cool, or neutral) using channel-weighted RGB analysis
  3. Determine your color season via HSL lightness quadrant mapping
  4. Match you to real foundation shades from a database of 70+ products (Fenty, MAC, NARS, Estée Lauder)
  5. Virtually try on foundation using AI-powered makeup rendering

6. Recommend outfit colors from your seasonal palette

How We Built It

Tech Stack

  • React Native (Expo SDK 54) with TypeScript
  • Perfect Corp YouCam API for AI skin tone analysis & virtual try-on
  • Custom algorithm for color season classification and foundation matching ### The Algorithm Undertone Detection — we compare green vs blue channels from the detected skin hex: $$ \text{Undertone} = \begin{cases} \text{Warm} & \text{if } G > B \times 1.04 \ \text{Cool} & \text{if } B > G \times 1.04 \ \text{Neutral} & \text{otherwise} \end{cases} $$ Season Classification — we map undertone × HSL lightness into quadrants: | | Light (L ≥ 45) | Dark (L < 45) | |---|---|---| | Warm | Spring | Autumn | | Cool | Summer | Winter | | Neutral | Spring | Winter | Foundation Matching — weighted Euclidean distance in RGB space, prioritizing green (×4) over blue (×3) over red (×2) to match human visual perception: $$ d = \sqrt{2\Delta R^2 + 4\Delta G^2 + 3\Delta B^2} $$ ### Architecture Home → Scan → Results → Profile │ ▼ Perfect Corp API Local Algorithm ┌──────────────┐ ┌──────────────┐ │ Skin Tone │──────▶│ Undertone │ │ Skin Analysis │ │ Season │ │ Makeup VTO │ │ Foundation │ └──────────────┘ │ Distance │ └──────────────┘ Global state is managed via React Context + useReducer with typed actions. Navigation uses manual screen switching (not React Navigation) for full control over the scan flow. --- ## Challenges We Faced
  • Image size constraints — the API requires images strictly between 320px and 1920px. We built an automatic JPEG resize pipeline using canvas (web) and ImageManipulator (native) that clamps both dimensions to exactly 321–1919px.
  • S3 CORS on web — the API's presigned S3 upload URLs lack Access-Control-Allow-Origin headers, causing fetch() to fail in browsers. We implemented an XMLHttpRequest fallback that strips forbidden headers and handles CORS-opaque uploads.
  • Face detection errors — when a face is too small in the frame, the API returns error_face_position_too_small. We surface this error directly to users with actionable guidance ("use a closer selfie") instead of silently falling back to demo data.

4. Virtual try-on visibility — AI-generated foundation effects can be subtle. We added a toggle to switch between the original photo and the try-on result for instant before/after comparison.

What We Learned

  • Hex is a universal skin language — representing skin tone as a hex color enables objective, reproducible matching across brands
  • Weighted RGB distance works — when tuned for human visual perception (green-sensitive), simple Euclidean distance in RGB space produces surprisingly accurate foundation rankings
  • CORS and presigned URLs don't mix well — browser-based uploads to S3 require thoughtful fallback strategies

- User feedback loops matter — showing why something failed (not just that it failed) dramatically improves the experience

What's Next

  • [ ] User accounts with scan history persistence
  • [ ] Expand foundation database with real-time product availability
  • [ ] Lipstick and blush virtual try-on
  • [ ] AR face tracking for real-time foundation preview
  • [ ] Social sharing of color season results
Share this project:

Updates