Lume

Log your beauty products with one photo. Find out which ones actually work for your skin. Build looks using only what you own.

Comment

Inspiration

Most people own a shelf of skincare and makeup and have no real idea what's working. The bottles pile up, routines get adopted on a friend's recommendation or a TikTok, and the feedback loop between "I've been using this for a month" and "is my skin actually better?" never closes. Existing tracker apps are passive logs — you type in what you own and the app just... remembers it. That's a filing cabinet, not an advisor.

Lume started from a simple question: what if your products could be measured against your actual skin? Perfect Corp's Skin Analysis gives clinical-grade metrics from a selfie. Pair that with what's in your collection, and you can finally answer "is this serum doing anything for me?" with data instead of vibes.

What it does

Lume turns a shelf of products into a personalized, measured beauty advisor.

  • Add a product in two photos. Snap the front — Lume removes the background and reads name, brand, subcategory, and shade in parallel. Snap the back — it OCRs the ingredients. If the box is long gone and OCR comes up empty, Lume searches Open Beauty Facts and falls back to a grounded web search, so you never hand-type thirty ingredients.
  • Scan your skin. A selfie returns 14 skin metrics — wrinkle, pore, acne, redness, moisture, dark circle, firmness, radiance, and more. Skin tone and face shape are extracted in the background and saved to your profile.
  • Get a verdict on every product. Gemini cross-references your skin metrics against each product's ingredients and tags every item works, neutral, or skip, with one sentence of reasoning anchored to a specific metric. No more guessing which products earn their spot.
  • Preview your skin in four weeks. From the verdict screen, Lume runs Skin Simulation on your top concerns and renders a before/after — a visual of where your current routine is taking you.

- Build me a look. Describe a vibe — "soft glam date night" — and Gemini picks the right subset of your owned makeup, assigns each product a slot, accounts for your face shape, then renders the look onto your selfie with Makeup VTO. It only ever uses products you already have.

How we built it

  • Frontend: Vite + React 19 + TypeScript (strict), Tailwind v4, React Router v7 data routes, Zustand for client state, TanStack Query for server state.
  • Backend: Supabase — Postgres, Auth, Storage, and Edge Functions (Deno).
  • AI / ML: Perfect Corp YouCam API for all the computer-vision-heavy work (Skin Analysis V2.1, Skin Tone Analysis, Face Analyzer, Skin Simulation, Background Removal, Makeup VTO). Google Gemini 2.5 Flash for vision OCR on labels and product fronts, verdict reasoning, look orchestration, and grounded ingredient search. The architecture keeps every API key server-side. The React client never touches a Perfect Corp or Gemini key — it calls thin Supabase Edge Functions that hold the secrets and proxy to the providers. Each function does one thing: analyze-skin, generate-verdict, generate-look, extract-ingredients, and so on, with shared CORS, client, prompt, and schema utilities factored into a _shared module.

A nice detail: adding a product fires multiple Gemini calls in parallel — front-of-package info extraction and background removal happen at once, so the product "develops" into a finished card in seconds rather than step by step.

Challenges we ran into

  • The pivot that started everything. Lume began on a completely different AI stack. When access to that provider's models fell through, the entire concept had to be rebuilt around APIs we could actually reach. That forced clarity: pick the provider first, design the product around what it does well. Perfect Corp's beauty-specific vision APIs ended up being a far better fit than the generic model we started with.
  • Ingredients are messy. OCR on a curved bottle in bad lighting fails constantly. Rather than make the user type ingredients by hand, we built a fallback chain: OCR first, then Open Beauty Facts, then a grounded Gemini web search. Most products resolve without the user lifting a finger.

- Async, key-safe AI calls. Perfect Corp's APIs are task-based — upload, create task, poll for the result — and the keys can't ship to the browser. Wiring that through Supabase Edge Functions, with clean error and timeout handling on calls that can take 15–30 seconds, was the backbone work that made everything else possible.

Accomplishments that we're proud of

Built solo in under two weeks. Four full AI-powered flows working end to end, integrating six Perfect Corp APIs plus Gemini across nine purpose-built Edge Functions. The "build me a look" flow chains Gemini reasoning into Makeup VTO so the result is grounded in what the user actually owns — not a generic filter. And the whole thing is genuinely useful: it's an app the developer would use on her own shelf.

What we learned

Constraining a model to a specialty (beauty vision) beats forcing a general model to do everything. Designing the product around the API's strengths — rather than bolting an API onto a fixed idea — produced something tighter and more demoable. On the engineering side: a clean Edge Function proxy layer pays for itself the moment you have more than two AI providers to coordinate.

What's next for Lume

  • Chrome extension — right-click any product image while shopping and instantly try it on your selfie (Makeup VTO) or simulate it on your skin (Skin Simulation). The entry point is already scaffolded.
  • Per-site adapters to pull product name, brand, and ingredients straight from retailer pages.
  • Longitudinal tracking — chart skin metrics over real time and correlate score changes with the exact products and dates they entered the routine.
  • Expiry and usage tracking so the collection becomes a true inventory.

Built With

  • deno
  • google-gemini
  • open-beauty-facts
  • perfect-corp-youcam-api
  • react
  • supabase
  • tailwind
  • tanstack-query
  • typescript
  • vite
  • zustand
Share this project:

Updates