AuraFit

AuraFit uses YouCam Skin AI, Apparel VTO, and local weather/UV data to give event-goers a personalized 7-day skincare prep countdown paired with color-matched virtual try-ons.

Comment

AuraFit: Intelligent Climate Adaptive Skin and Wardrobe Concierge

Inspiration

Preparing for defining life moments, whether an energetic traditional Owambe wedding in Lagos, a black tie gala in New York, or an executive tech summit in London, often brings unnecessary stress. We noticed two persistent problems in how people get ready:

  1. Skincare is reactive and blind to the local climate. People often begin aggressive treatments too close to an event, causing irritation or post-inflammatory hyperpigmentation, especially on melanin rich skin. Generic routines also ignore crucial atmospheric factors like high humidity, heat, and intense UV radiation.
  2. Wardrobe styling involves too much guesswork. Deciding whether a garment’s color complements your natural undertone or how a piece drapes on your silhouette is difficult without trying it on in person.

We built AuraFit to bridge this divide. Our goal was to design an intelligent personal concierge that unifies computer vision skin diagnostics, local climate forecasting, and virtual apparel fitting into a personalized 7 day preparation countdown.

What it does

AuraFit streamlines personal styling and event prep into an automated, science backed flow:

  • Flexible Portrait Capture: Takes inputs via live WebRTC camera streaming from your phone or webcam, direct image uploads, or public image links.
  • Skin Health Diagnostics: Evaluates hydration, redness, dark spots, fine lines, and dark circles while detecting natural undertones ($\text{warm}$, $\text{cool}$, or $\text{neutral}$) using Perfect Corp YouCam Skin AI.
  • Atmospheric Intelligence: Fetches real time temperature extremes and maximum UV index data ($UV_{\text{index}}$) across Nigerian metropolitan hubs (Lagos, Abuja, Enugu, Port Harcourt) and global cities.
  • Melanin Aware 7 Day Protocol: Compiles an adaptive daily skincare schedule with targeted actives (gentle Mandelic Acid, Tranexamic Acid, lipid barrier recovery) and invisible chemical SPF 50+ sunscreens that leave zero white cast.
  • Live Color Harmonization: Uses The Color API to extract visual swatches, shade titles, and contrast luminance values that complement the user's specific undertone.
  • Multi Category Virtual Try On: Lets users virtually fit full body outfits, tops, skirts, traditional Gele headwear, and footwear directly on their silhouette using YouCam Clothes AI.

How we built it

┌─────────────────┐       ┌──────────────────────┐       ┌──────────────────────┐
│  Next.js 16 UI  │ ────> │ API Route Controller │ ────> │ Open-Meteo & Color   │
│  (React/Tailwind)│ <──── │ (/api/event-plan)    │ <──── │ External APIs        │
└─────────────────┘       └──────────┬───────────┘       └──────────────────────┘
                                     │
                                     ▼
                          ┌──────────────────────┐
                          │ YouCam S2S Gateway   │
                          │ (/task/skin-analysis)│
                          │ (/task/cloth-v4)     │
                          └──────────────────────┘
  • Frontend Architecture: Built with Next.js App Router and TypeScript, styled with a custom Tailwind CSS luxury design system moving across blush pink (#FDF2F0), warm rose brown (#C68B7B), rich terracotta (#A85A48), and deep espresso brown (#2D1812).
  • Media Streaming: Used HTML5 Canvas and browser media streams (navigator.mediaDevices.getUserMedia) for zero latency selfie snapshots.
  • Diagnostic & Try On Integration: Built a modular TypeScript backend connecting directly to Perfect Corp YouCam Server to Server (S2S) cluster endpoints with an asynchronous polling loop and timeout management.
  • Color Intelligence Engine: Connected to The Color API and evaluated photometric luminance:

$$Y = 0.299R + 0.587G + 0.114B$$

This ensures every generated color card satisfies accessible contrast thresholds ($Y > 0.55$).

  • Data & Auth Layer: Integrated Supabase for secure user authentication, lookbook saving, and historical consultation storage.

Challenges we ran into

  • S2S API Gateway & Polling Structure: Early tests returned 404 and 405 errors due to differences between public console endpoints and active server clusters. We resolved this by inspecting raw network responses, connecting to the active -01 cluster, and querying task status directly at /task/cloth-v4/{taskId} using lowercase status flags (task_status: "success").
  • Strict Payload Validation: The YouCam virtual try on engine strictly mandated specific parameter schemas (such as garment_category: "full_body" and change_shoes: true). Passing local base64 images directly caused network drops, so we added fallback mechanisms, abort controllers, and schema validation.
  • Melanin Inclusive Formulation Logic: Standard beauty tech tools often recommend harsh scrubs or chalky mineral sunscreens that cause white cast on dark complexions. We calibrated our engine to recommend gentle chemical exfoliants, barrier restoring ceramides, and invisible chemical filters for high UV conditions ($UV_{\text{index}} \ge 6$).

Accomplishments that we're proud of

  • True Cross Domain Personalization: Created an end to end pipeline linking climate data, facial diagnostics, color science, and generative garment try on.
  • Inclusive Cultural Representation: Built tailored support for African events and garments (such as Nigerian Owambe Asoebi, brocades, and architectural Gele crowns) alongside international galas and summits.
  • Resilient Asynchronous Engine: Handled multi second computer vision jobs cleanly with instant feedback and robust error recovery so the user interface remains fast and responsive.
  • Accessible Luxury Design: Crafted a high contrast, responsive aesthetic that passes accessibility checks while maintaining an elevated fashion tech feel.

What we learned

  • Orchestrating Complex AI Services: Managing multiple asynchronous generative APIs requires clear loading states, defensive fallback bridges, and non blocking server architecture.
  • Context Aware Product Thinking: Personalization goes beyond basic filters; factoring in geographical weather, cultural fashion norms, and skin physiology makes recommendations significantly more practical.
  • Full Stack Engineering Harmony: Pairing server side API protection with client side camera streaming creates a secure, low latency interactive experience.

What's next for AuraFit

  • Direct E-Commerce & Designer Checkout: Partner with African fashion houses, local designers, and skincare retailers to enable one click purchasing directly from the virtual fitting room.
  • Automated Daily Reminder Engine: Integrate SMS and push notifications that deliver daily skincare tasks and weather alerts throughout the 7 day countdown.
  • Augmented Reality Live Fitting: Expand beyond static try on snapshots into real time WebGL/WebXR video try on for dynamic garment drape and lighting simulation.
  • Community Lookbook Sharing: Enable social collaboration where bridal parties, friends, and event attendees can coordinate matching color palettes and outfits in shared workspaces.

Built With

Share this project:

Updates