Inspiration
Most skincare purchases happen at one moment: in front of a shelf (or a search page) of 40 serums, often right after a breakout. Skin-analysis apps give you a score and stop there. Shops give you a search box and let you guess. The two never connect, so people buy by hype, stack products that clash, and return foundation that was the wrong shade. We wanted one flow that answers "what is going on with my skin, what should I actually do, what should I buy, and how will it look on me?" in under a minute.
What it does
- Snap a selfie (or use the sample). Pick a budget and goals such as "calm redness" or "clear breakouts", plus sensitive or fragrance-free filters.
- YouCam Skin Analysis scores 14 concerns (redness, acne, pores, oiliness, texture, dark circles, hydration and more), and YouCam Skin Tone Analysis reads the skin colour.
- The SkinSync agent (an LLM with tools) explains the top 3–4 findings in kind, plain language. It then shops the catalog with batched tool calls and builds a morning and evening routine within your budget: sunscreen every morning, one targeted night treatment, no clashing actives. It also matches your foundation shade to your measured skin tone.
- YouCam Makeup Virtual Try-On renders the recommended look (Fresh Dewy, Soft Glam or Office Matte) on your photo, using the exact lipstick, blush and foundation SKUs, with a before/after slider. Swap shades and re-render.
- Refine in plain English, for example "I'm pregnant, no retinoids" or "keep it under $60" or "bolder for a party". The agent re-plans and the try-on updates.
- Add the routine or the look to the bag in one tap. A "How the agent got here" panel shows every YouCam call and agent tool call.
Who it's for: shoppers who feel overwhelmed by skincare choices, and beauty retailers who want higher conversion, fewer returns (shade-matched base, try before you buy) and bigger baskets (a full routine instead of a single SKU).
How we built it
- Backend: FastAPI with an async YouCam v2 client (
httpx). Every feature follows the documented flow:POST /s2s/v2.0/file/{feature}→PUTto the pre-signed URL →POST /s2s/v2.0/task/{feature}→ pollGET /s2s/v2.0/task/{feature}/{task_id}untilsuccess/error, withAuthorization: Bearer <API key>. Skin Analysis and Skin Tone run in parallel. Images are EXIF-corrected and resized to each feature's documented limits. - Skin Analysis: all 14 SD
dst_actionswithformat: "json", normalised into one score model. - Agent: Groq
openai/gpt-oss-120bwith OpenAI-style tool calling. A batchedsearch_productstool means a plan usually takes two model turns and about 3 seconds. The server validates every SKU (hallucinated ids are dropped) and runs a budget self-check. A deterministic rule engine takes over if the LLM is unavailable. - Makeup VTO: a small compiler turns look + chosen SKUs into a YouCam
effectsarray (skin_smooth,foundation,blush,lip_color). - Frontend: one responsive page in vanilla JS with a score ring, concern bars, routine tabs, a before/after slider and a bag drawer.
- DEMO_MODE: canned responses with the same shapes as the live API, so judges can run the whole flow without a key.
- Tests: pytest plus respx mocks of the YouCam contract and the full app flow.
Challenges we ran into
- Turning scores into advice without overclaiming. YouCam's
ui_scoreis "higher is healthier", so we framed results kindly and cosmetically, never diagnostically. - Free-tier LLM limits. Groq's 8k tokens per minute ruled out a chatty agent, so we batched tools, trimmed payloads and added a rule-based fallback.
- The makeup VTO schema is detailed. We built an effects compiler that always emits valid combinations.
- Different size rules per feature. Each YouCam call gets its own resized copy.
Accomplishments that we're proud of
- Three YouCam APIs chained into one coherent shopping journey: diagnose, explain, prescribe, match the shade, try on, buy.
- A real agent loop with tool use, SKU validation and a budget self-check in about 3 seconds.
- Try-on that shows the exact products in your basket, not a generic filter.
What we learned
- How the YouCam v2 file→task→poll pattern works across features, and how much the input photo matters.
- Agents become more reliable when you validate on the server and feed errors back.
- In beauty commerce, the try-on moment is where confidence turns into a purchase.
What's next for SkinSync
- Re-scan check-ins every 2–4 weeks to chart each concern's score.
- YouCam AI Skin Simulation to preview expected improvement.
- Live camera try-on with the YouCam JS Camera Kit.
- A real merchant catalog feed and checkout; multilingual explanations (Hindi first).