Testing access: Judges can find the live-preview access code and testing instructions at https://github.com/Alori-Daniel/deeper-tones after accepting the GitHub repository invitation.

Inspiration

Amara is 29 and lives in Lagos. For three years she has used a cream from the market that keeps her skin even. She has never read the small print on the tube, which says it contains clobetasol propionate, one of the strongest steroids there is. Every time she tries to stop, her skin flares worse than before, so she goes back to it. She is afraid to stop.

Amara is a persona, but her story is common. Creams with strong steroids are sold over the counter as "toning" or "lightening" creams. Stopping them suddenly can cause a rebound flare, and that flare keeps people on them.

Most skincare apps don't help her. They are built and tested mostly on lighter skin, and they answer every problem by selling another product. Nobody looks at the shelf she already has and tells her what is helping, what is hurting, and how to stop the cream safely.

We built Deeper Tones to be that coach. It is made for darker skin first, and it works from what she already owns.

What it does

  1. Picks one goal. The user chooses a goal, such as even tone or fewer breakouts, and answers a short safety screen.
  2. Reads the face, with consent. The face photo is optional. Perfect Corp's CameraKit guides the capture live on the phone (framing, pose and lighting), so the photos are consistent from week to week. YouCam's skin analysis then describes what the photo shows. It never gives a diagnosis.
  3. Reads the shelf. The user photographs the products they own. Deeper Tones recognises each one and reads its ingredient list, even a medicine's "Composition" line. It then sorts the shelf into plain groups:
    • helping the goal;
    • doing the same job as another product;
    • blocking the goal;
    • clashing with another product;
    • likely causing the problem.

Each product comes with a one-line reason.

  1. Steps down, never stops cold. When a product contains a strong steroid, Deeper Tones doesn't say "stop now". It builds a dated step-down plan instead: every other day, then once a week, then stop. It also says what to expect and when to pause.
  2. Builds the routine from her own shelf. Products that already work stay in the routine. The app only suggests something new to fill a real gap, such as sunscreen. It never replaces a product that works with one from the catalogue.
  3. Changes the routine as her skin changes. Daily reminders, skin logs and check-ins feed back into the routine. If her skin stings, she can pause a step. If something looks serious, the app tells her to see a professional.

There are three things it never does: it never diagnoses, it never tells her to stop a steroid cold, and it never sells her a whole new shelf.

How we built it

Mobile app

  • React Native with Expo SDK 56, Expo Router and TypeScript.
  • A custom Expo module in Kotlin and Swift that wraps Perfect Corp CameraKit. Its live quality events drive the capture screen on both Android and iOS. If the module isn't available, the app falls back to Expo Camera.
  • Zustand for session and onboarding state. Axios with access and refresh tokens, stored in SecureStore.
  • Server-sent events, with polling as a fallback, for face scans, product scans, verdicts and routines.
  • Expo Notifications, Expo Updates and EAS Build.

API

  • Hono on Cloudflare Workers, with D1 and Drizzle. The code is layered: routes call services, services call repositories, and every third party sits behind its own provider.
  • YouCam skin analysis, set to the user's goal. It returns the overlays, skin type and skin age.
  • Claude recognises products from photos, reads ingredient labels and writes the explanations. Haiku does the internal steps and Sonnet writes what the user reads. We track the cost of every AI call.
  • A reviewed ingredient rule set sets the safety floor. The AI can add caution on top of the rules, but it can never remove a safety flag the rules raised.
  • Evaluation sets for product matching, verdicts and routines. A change ships only if it meets the bar we set for that set.

Challenges we ran into

  • A native camera SDK in a managed Expo app. We wanted one capture experience on Android and iOS. To get it, we built a local Expo module, passed CameraKit's quality events up to TypeScript, and kept a fallback camera for when the module isn't available.
  • Real products are messy. Our first product recogniser rejected the very cream this project is about: it filed medicated creams as medicine, not skin care. The label reader also missed ingredient lists printed as "Composition: Clobetasol Propionate 0.05% w/w". We fixed both by changing the general rules, not by adding a special case for one product, and we tested the fix with products outside our evaluation sets.
  • Being useful without diagnosing. The app has to say "this cream is likely the problem" without making a medical claim. We use calm wording, explicit consent and a safety floor in code, and the app points the user to a professional when a case needs one.
  • Slow analysis on mobile networks. Face and shelf analysis take time. We used live event streams with polling fallbacks and honest progress states, so the app never says a result is ready before the server does.

Accomplishments that we're proud of

  • CameraKit makes real capture decisions in the app, rather than sitting in a separate demo.
  • The face read, the goal and the user's own shelf work together as one coaching experience.
  • The step-down plan for steroid creams, with real dates. As far as we know, no other skincare app offers this.
  • Routines built from the products the user already owns. When the shelf changes, the verdict is recalculated.
  • A feedback loop: check-ins and skin logs change the routine over time, instead of leaving a one-off report.
  • The app handles unreadable labels, products added by hand, retakes, a denied camera permission, dropped event streams and returning users.

What we learned

  • Capture quality matters as much as the analysis. Live camera feedback gives the analysis better photos to work with.
  • The hard part of AI skincare is the safety around it, not the model. That means consent, explanations, and rules that only ever add caution.
  • People need a coach that adapts, not a one-off report. For someone like Amara, the most valuable thing is often not a new product. It's a safe plan to stop the one that is hurting her.

What's next for Deeper Tones

  • Test with more people across the full range of deeper skin tones, and measure whether guided capture keeps the analysis consistent across lighting and devices.
  • Have dermatologists review the step-down plans and the educational content.
  • Cover more products and ingredients, starting with what is sold in markets in Nigeria.
  • Launch in Nigeria, the UK and the US, with progress comparisons over time and routines that respond even more closely to skin logs.

Try it

Built With

Share this project:

Updates

Submission history