meleon

Inspiration

Everyone has stood in front of a cart with the same shirt in three colors and no idea which one is theirs. "It looked different on me" tops apparel return reasons, and the existing answer — personal color consulting — costs real money, varies by consultant, and ends with a paper swatch you can't shop with. We wanted the consult to be a measurement, and the recommendation to be proof on your own face, not a promise.

What it does

meleon is one continuous experience where Skin AI decides what's worth trying on, and Apparel VTO proves it — the two track capabilities operate as a single pipeline, not as separate features.

One selfie → your skin's actual color coordinates (YouCam Skin Tone + Fitzpatrick + Face Attribute analysis) → a 12-tone season palette with a plain-language stylist interpretation → then the palette codes everything downstream:

1. Hair color — your palette is snapped to the nearest real, purchasable dye shades (726-shade embedding space), a colorist persona explains the top picks, and YouCam Hair Color renders them on your photo.
2. Haircut — your measured face shape (never guessed by us — measured by Face Attribute analysis) drives a rule table that ranks a hairstyle catalog. Renders are chained cut → color so the preview still wears the dye you already applied.
3. Fashion — the store re-ranks itself for your skin: every product gets a CIEDE2000 harmony score against your palette snapshot, and Apparel VTO (cloth-v3) renders the winning colorways on you. If you skipped the upper-body photo, your styled hair render becomes the try-on source — the look stays yours end to end.

The finished look is shareable as a public snapshot link, and the catalog is real: products are fulfillable through dropshipping with checkout, orders, and shipping management built in. An AI stylist agent (SSE chat) can drive the same studio — same state, same caches, same prices as the screens.

How we built it

YouCam S2S APIs: skin-tone-analysis + fitzpatrick-scale-analyzer + face-attr-analysis (diagnosis batch, parallel), hair-color (arbitrary-hex palettes — we send real dye shades, not presets), hair-transfer v2.1 (cut→color chaining), cloth-v3 (apparel VTO with change_shoes: false).

Our color engine — sRGB→CIELAB normalization, 3-axis classification (undertone/value/chroma) → 12-tone season, CIEDE2000 harmony scoring. Classification thresholds are calibrated against the measured distribution of the vendor's camera pipeline, not textbook constants — production data showed every user collapsing into one season under literature thresholds, so we re-anchored the boundaries to measured gaps. This engine, not the API, is what turns measurements into a shoppable ranking.

Two-axis hair logic — palette decides COLOR, face geometry decides CUT. The two axes never mix; the color axis literally cannot see the style axis's inputs.

Webhook-first async — the vendor rate limit is 250 req/300s per token; one second of polling for a single session (300 req) would exceed it. So: HMAC-verified webhooks, idempotent by webhook-id, with polling only for local dev.

Instant result mirroring — vendor render URLs expire in 2 hours. Every result is downloaded and mirrored to our storage inside the receiving transaction; clients only ever see our copies, so a judge opening a saved look days later sees images, not 404s.

LLM prose, measured facts — stylist/colorist/designer interpretations are persona-generated in 7 locales from fact sheets (measurements + rule outputs). If generation fails we say so and offer retry — we never fill the gap with invented prose.

Challenges we ran into

Photo-spec conflict: diagnosis wants a face at ~60%+ of frame width; VTO demands visible shoulders. One photo can't satisfy both, so the flow uses a cropped selfie for diagnosis and later renders as try-on sources — with client-side framing pre-checks so vendor rejections happen before upload, not after.

task/delete destroys input files: the cleanup API deletes a task's input files too, which silently killed every follow-up render reusing the same upload. We measured it (0 successes / 9 failures on already-deleted files), moved deletion to asset-retirement time, and added an ESLint guard so no new call site can reintroduce it.

The hair-color engine preserves lightness: honey blonde on near-black hair renders as almost no change (PSNR 45.9 dB — we measured it). Instead of generatively brightening the photo (which would fake the measurement), we predict expression: shades brighter than the measured hair lightness by more than a realistic no-bleach lift are labeled "subtle on your hair" before the user spends anything.

Unit budgeting: 1,000 free units for everything — so tests run on vendor-contract fixtures, every failure path was measured for refund behavior (the vendor refunds engine failures; timeouts are the only true loss), and a budget floor blocks new diagnoses before the reserve is touched.

Accomplishments that we're proud of

• A measurement product that never redraws the user: crop/rotate/resize only. We built and then deliberately removed a generative photo-fix path when it altered identity — the honest alternative (targeted error guidance in 7 locales) shipped instead.
• Recommendations that survive adversarial review: every ranking claim (labels, reasons, footnotes) is machine-checked against the attributes it speaks about — 9,600 combination sweeps, zero contradictions.
• A full commerce loop on top of the demo: credits ledger, PayPal physical-goods payments, dropshipping fulfillment with real supplier orders placed in sandbox E2E.

What we learned

• Vendor names are not contracts — read the ledger, not the label. (Token types said "PAYG"; the actual redeem records said otherwise. Template IDs documented as numbers arrived as strings.)
• Fixtures must model the vendor's contract, not our convenience — twice, a fixture that was stricter or looser than the real API hid a product-breaking bug.
• Absolute literature thresholds don't survive contact with a real camera pipeline; calibrate classifiers on measured distributions and version the engine.

What's next for meleon

• B2B storefront widget — the harmony-ranking engine drops into any product list; returns go down, conversion goes up.
• Mobile app — the BFF contract is already client-neutral (Bearer sessions, absolute URLs, one task-status contract); a mobile client reuses it unchanged.
• MCP agent surface — the studio tools already run behind a ports interface; exposing them over MCP makes any agent a personal shopper.

A note on the demo footage

The people in our video and screenshots are AI-generated images, not real users. We did not want a real person's face in a public submission, and swapping a stock face onto someone else's measurements would have been worse — it would show one person's skin coordinates labeled as another's. So we generated two synthetic subjects and ran the real pipeline on them from scratch: upload, framing check, three analysis tasks, hair color, chained cut, apparel try-on, share, checkout. Every number and every render you see is genuine API output measured from those photos. The same disclosure appears on the closing card of the video.

Built With

• anthropic
• docker
• drizzle-orm
• firebase
• next.js
• node.js
• paypal
• postgresql
• react
• tailwindcss
• typescript
• youcam-api