SkinMatch AI

AI-powered skin analysis app that delivers personalized skincare routines for every skin type and climate, starting with the world's most underserved tropical skin types.

Comment 2

✨ SkinMatch AI: DevPost Project Story

Personalized skincare for every skin, every climate

Hackathon Next.js TypeScript Vercel

Perfect Corp Crusoe Cloud TrueFoundry

🚀 Live Demo · 📹 Demo Video · 📄 DevPost

Inspiration

It started with a frustrating conversation.

A friend of mine (21, from Indonesia, oily acne-prone skin) had spent over $80 on skincare products that all broke her out. She'd tried three different AI skincare apps. Every single one recommended products built for dry, temperate-climate skin. None of them understood her skin.

That conversation stuck with me. I started digging into the data:

  • 63% of people don't actually know their real skin type, meaning the majority are buying products that don't work, or actively damage their skin (Skin Trust Club / Labskin, 2022)
  • 90% of consumers feel frustrated trying to find skincare that works (Drive Research, 2025)
  • 68% of consumers want personalized skincare, but current AI tools fail to serve tropical and diverse skin types (Business Research Insights, 2026)

The global skincare market is worth $189.3 billion and growing, yet the AI tools in this space are almost entirely built and trained on Western, 4-season climates. There's a massive, underserved population: Gen Z in Southeast Asia, Africa, and Latin America with tropical, oily, humid-climate skin types that behave completely differently.

Nobody had built an AI skincare advisor for them.

That's what inspired SkinMatch AI.

What it does

SkinMatch AI is a web application that gives anyone a hyper-personalized skincare routine in under 30 seconds, completely free, no sign-up required.

Here's the full experience:

  1. 📸 Snap or upload a selfie (works on any device, including mobile camera)
  2. 🔬 Perfect Corp's AI Skin Analysis scans the photo using computer vision, detecting:
    • Skin type (oily, dry, combination, normal, sensitive)
    • Skin tone (light, medium, deep)
    • 8 active skin concerns: acne, dark spots, wrinkles, pores, texture, redness, dark circles, oiliness
  3. 🤖 Nvidia Nemotron LLM (via Crusoe Cloud) generates a fully personalized routine:
    • Morning routine (step-by-step with product types and application order)
    • Night routine (targeted to repair concerns while you sleep)
    • 6 product recommendations with real brand names, price ranges, and why each one fits your skin
  4. 📊 Skin Report Card: a beautiful, shareable summary of everything detected and recommended

The key differentiator: SkinMatch AI is built specifically for tropical, diverse skin types. Climate-aware prompting, oiliness-prioritized skin type inference, and product recommendations that are globally accessible, not just available at a US pharmacy.

How we built it

Built solo in 3 days using a fully serverless, zero-cost stack.

Frontend

  • Next.js 15 with App Router and TypeScript (SSR + API routes in one project)
  • Tailwind CSS v4 for utility-first styling with a custom glassmorphism design system
  • Framer Motion for fluid animations: staggered card reveals, floating orbs, parallax scroll, slide transitions between camera/upload/preview modes

AI Skin Analysis: Perfect Corp API

Implemented a 4-step async flow:

Step 1: POST /s2s/v2.0/file/skin-analysis  ->  get presigned S3 URL + file_id
Step 2: PUT image directly to S3           ->  upload bypasses Perfect Corp servers
Step 3: POST /s2s/v2.0/task/skin-analysis  ->  create analysis task, get task_id
Step 4: GET poll every 2s until success    ->  extract skin scores + concerns

The most complex part was parsing the raw score data into meaningful skin profiles and writing the inferSkinType() logic that correctly identifies tropical oily/combination skin patterns.

LLM Layer: 5-Level Resilient Fallback Chain

This is the architectural heart of SkinMatch AI. Every request flows through:

L1: TrueFoundry AI Gateway  ->  routes to Nemotron -> Llama -> Qwen (priority order)
L2: Nemotron direct         ->  Crusoe Cloud Managed Inference (bypass gateway)
L3: Llama 3.3-70B direct    ->  Crusoe Cloud
L4: Qwen3-235B direct       ->  Crusoe Cloud
L5: Static JSON rules       ->  pre-written routines by skin type, zero API dependency

📸 Skin Analysis Results (Perfect Corp)
              ↓
   ┌─────────────────────────┐
   │  L1: TrueFoundry        │  ← AI Gateway (routing + monitoring)
   │      AI Gateway         │    Priority: Nemotron → Llama → Qwen
   └────────────┬────────────┘
                │ ❌ Gateway down
                ↓
   ┌─────────────────────────┐
   │  L2: Nemotron direct    │  ← Crusoe Cloud Managed Inference
   │  (Crusoe Cloud)         │    hack-crusoe/Nemotron-3-Nano-30B
   └────────────┬────────────┘
                │ ❌ Nemotron unavailable
                ↓
   ┌─────────────────────────┐
   │  L3: Llama 3.3 direct   │  ← Crusoe Cloud
   │  (Crusoe Cloud)         │
   └────────────┬────────────┘
                │ ❌ Llama unavailable
                ↓
   ┌─────────────────────────┐
   │  L4: Qwen direct        │  ← Crusoe Cloud
   │  (Crusoe Cloud)         │
   └────────────┬────────────┘
                │ ❌ All LLMs down
                ↓
   ┌─────────────────────────┐
   │  L5: Static JSON        │  ← Pre-written routines by skin type
   │  Rules Engine           │    Zero dependency, always works
   └─────────────────────────┘
              ↓
   ✅ User always gets a response
   💰 Total infra cost: $0

The TrueFoundry Gateway handles routing, automatic retries with exponential backoff, and real-time observability. If the gateway itself goes down, we fall through to direct Crusoe calls. If all LLMs are unavailable, users still get a complete, useful routine from the static engine.

The user always gets a response. No loading spinners that never resolve. No error screens.

Nemotron-specific Engineering

Nemotron is a reasoning model. Its API can return content: null when the response is in the reasoning field instead. This took hours to debug. The fix: always check both choices[0].message.content and choices[0].message.reasoning, with JSON extraction from whichever contains valid data.

Challenges we ran into

🔴 Nemotron returns content: null

The biggest debugging session of the hackathon. Nemotron is a reasoning model that sometimes puts its entire output in a reasoning field while returning null for content. Our initial implementation was silently failing and falling through to L2 every time. Fix: added a dual-field parser that checks both fields and extracts the first valid JSON block found.

🔴 TrueFoundry Virtual Model ID format

The correct format for TRUEFOUNDRY_MODEL_ID is group/model (e.g. skinmatch-ai/skinmatch-routine), not just the model name alone. This took significant trial-and-error to discover. The error was silent: the gateway would simply return 404 with no clear message.

🔴 Qwen model naming on Crusoe

The correct model identifier is Qwen/Qwen3-235B-A22B-Instruct-2507, discovered only after calling GET /v1/models on the Crusoe API and comparing against what was listed. The documentation and the actual deployed model name differed.

🔴 Mobile photo upload on Android

Photos uploaded from Android phones were hitting the Perfect Corp API in unexpected ways. Large file sizes and varying JPEG quality caused inconsistent results. Fixed by implementing a canvas-based JPEG normalization pipeline: every uploaded photo is drawn to a canvas, resized to max 1920px on the longest side, and exported as JPEG at 85% quality before sending to the API. This also solved HEIC compatibility for iPhone users.

🔴 Tailwind CSS v4 breaking changes

bg-gradient-to-br is now bg-linear-to-br in Tailwind v4. Framer Motion's ease: "easeOut" string isn't valid in transition props and requires either removing the ease or using a bezier array. These silent failures wasted several hours of UI debugging.

🔴 .next/trace owned by root

Running Next.js as root at some point caused the .next/trace file to be owned by root, breaking all subsequent builds as a non-root user. Required sudo rm -rf .next to resolve.

Accomplishments that we're proud of

✅ 5-Level Fallback: Zero Downtime Architecture

This isn't just a demo feature. It's production-grade resilience. During testing, we intentionally killed each LLM layer one by one. Every time, the app silently fell through to the next level and returned a complete, useful response. The user experience was completely uninterrupted.

✅ Real Computer Vision, Not a Quiz

90% of skincare apps ask you to self-report your skin type. The problem: 63% of people are wrong about their own skin type. SkinMatch AI uses actual computer vision. Perfect Corp's SDK analyzes 8 biometric skin markers from a single photo. The results are objective, not self-reported.

✅ Tropical Skin Intelligence

The skin type inference logic was hand-tuned for tropical skin patterns: prioritizing oiliness and acne scores, correctly identifying combination skin in humid climates, and flagging redness + low oiliness as sensitive rather than dry. This matters because recommending a heavy moisturizer to someone with oily tropical skin is actively harmful.

✅ $0 Infrastructure Cost

The entire stack runs on hackathon sponsor credits. Vercel for hosting, Perfect Corp for CV, Crusoe for LLM inference, TrueFoundry for the gateway layer. A production-grade AI application at zero operational cost.

✅ End-to-End in 3 Days, Solo

From blank Next.js project to deployed, tested, end-to-end working application in 3 days, one developer, three sponsor API integrations, five LLM fallback layers, full mobile support, and polished Framer Motion UI.

What we learned

Reasoning models require different parsing logic. Nemotron and similar reasoning LLMs don't always return their output in message.content; sometimes it's in message.reasoning. Any production LLM integration needs to handle this gracefully.

Resilience is a product feature, not just an engineering concern. The fallback chain isn't just there for reliability, it's part of the pitch. Being able to demonstrate live that killing the primary LLM doesn't break the user experience is a powerful differentiator for the TrueFoundry challenge.

Mobile-first is non-negotiable for a skincare app. The primary use case is someone in a bathroom, on their phone, taking a selfie. Every design decision, from the canvas JPEG pipeline to the full-viewport camera UI, had to work flawlessly on a mobile browser with no native app install.

LLM prompt engineering for structured output is underrated. Getting Nemotron to reliably return a consistent JSON schema with morning/night routines, step-by-step instructions, and product recommendations every time, across multiple fallback models with different training, required significant prompt iteration and JSON extraction logic.

Tropical skin is a real engineering problem. Writing skin type inference logic that correctly handles oily, humid-climate skin required understanding dermatology, not just coding. The threshold values in inferSkinType() aren't arbitrary. They're calibrated to surface the right results for the demographic we're serving.

What's next for SkinMatch AI

SkinMatch AI has a clear path from hackathon project to sustainable product:

🌴 Climate Mode (Next Sprint)

Toggle between Tropical, Temperate, and Cold climate modes. The LLM prompt adjusts humidity, UV index, and seasonal factors, dramatically changing which products and ingredients are recommended. Someone in Jakarta needs a completely different routine than someone in Stockholm, even with identical skin scores.

💰 Budget Filter

Filter all product recommendations by price tier: $ (drugstore), $$ (mid-range), $$$ (premium). Democratizing access to good skincare means respecting that not everyone can spend on La Mer.

📈 Progress Tracker

Save and compare skin analysis results over time. See whether your routine is actually improving your acne score, reducing oiliness, or clearing dark spots, with objective CV data instead of self-assessment.

🤝 B2B White-Label API

License the climate-aware skin analysis layer to skincare brands (CeraVe, Innisfree, The Ordinary) to embed directly on their product pages. A customer landing on a CeraVe page could get instant personalized product recommendations without leaving the site.

🌏 Expanding the Training Data

Partner with dermatologists in Southeast Asia and Africa to validate and improve the skin type inference logic for underrepresented skin tones and tropical skin conditions.

The goal: become the default AI skincare advisor for the 4 billion people in tropical climates who've never had a tool built specifically for them.

Built for DevNetwork AI + ML Hackathon 2026 - Challenges: Perfect Corp + Crusoe Cloud + TrueFoundry

Built With

  • crusoe-cloud-managed-inference
  • llama-3
  • next.js
  • nvidia-nemotron
  • perfect-corp-ai-skin-analysis-api
  • qwen
  • react
  • supabase
  • tailwindcss
  • truefoundry-ai-gateway
  • typescript
  • vercel
Share this project:

Updates