Shelf Check

Inspiration

We build online stores for small businesses, and beauty shops keep telling us the same thing: shoppers stand in front of forty serums, ask "which one is for me?", and leave without buying when nobody is there to answer. Skin analysis apps exist, but they recommend from their own brand catalog, not from the shelf the shopper is standing in front of.

What it does

The shopper takes a selfie (or a judge taps "Use sample selfie"). Shelf Check sends it once to the YouCam File API and runs two YouCam tasks on it: the Skin Analysis API measures eight concerns (breakouts, pores, redness, oiliness, dehydration, fine lines, dark circles, dark spots) and the Fitzpatrick Skin Type API classifies the skin from type I to VI. The app shows the skin map (tap a concern to see the API's own mask over your photo), the overall score, the estimated skin age and the skin type, with one line on what that type means: on types I-II sun damage shows first, so fine lines, redness and spots weigh more; on types IV-VI the marks left by breakouts (post-inflammatory hyperpigmentation) and visible dryness lead, so those weigh more. Then it picks the three products, from this store's own catalog, that cover the most of the shopper's weighted need. Every pick says why ("For: Dehydration, Redness"), so the shopper trusts it and the store sells what it actually has on the shelf.

How we built it

The YouCam flow is File API (pre-signed upload) -> skin-analysis task (SD, 8 concerns, mask overlay, 12 units) and fitzpatrick-scale-analyzer task on the same file, run in parallel -> poll both until success. For each concern, need = (100 - raw_score) x skin-type weight; weights never go below 1, so a measured need is never hidden, the type only says which needs matter more for that skin (e.g. dark spots x1.3 on type V, fine lines and redness x1.2 on type I). Each catalog item scores the sum of the needs it treats, ties go to the cheaper item. The skin type is advisory: if its task fails (face angle, lighting) the shopper still gets the skin map weighed as measured. The store's catalog is a plain catalog.json (sku, name, price, treats), so any shop plugs in its own shelf. The live app runs on Cloudflare Workers (worker.js) with a dependency-free Python twin (server.py) and a parity test that checks the Worker ranks exactly like the Python reference for all six skin types.

Challenges we ran into

The engine rejects faces that are too small (error_src_face_too_small): the face has to fill more than 60% of the frame, so the sample photo had to be cropped. The mask URLs returned by the task are signed and expire in two hours, so the demo serves the captured masks from the app itself instead of dead links.

Accomplishments that we're proud of

A live public app that answers in about seven seconds with a real YouCam call, and a recommendation that is explainable line by line: every pick shows which measured concerns justified it.

What we learned

The raw scores are precise enough to rank products directly, and turning "score" into "need" makes the recommendation feel like advice instead of a diagnosis.

What's next for Shelf Check

Import the catalog straight from Nuvemshop and Shopify product tags, let the store tune the skin-type weights for its own range, add the Apparel VTO step for the clothing side of the same store, and an in-store kiosk mode for a tablet next to the shelf.

Built With

• cloudflare-workers
• javascript
• perfect-corp
• python