Landing Page
Architecture Diagram
Skin Health dashboard with the concern scores
Concern for skin health
Targeted Recommendation to improve the skin health
Outfit Recommendation according to the events throw closet
Outfit Builder

MirraAI — your AI lifestyle mirror

Two photos. One AI agent. A transparent Proof Card backs every recommendation before you spend a cent.

Most beauty/fashion apps stop at a score. MirraAI is an AI agent that turns those scores into a decision: what to apply, what to wear, what to buy, and what to skip — calling 9 Perfect Corp APIs, real-time weather, your closet, and live shopping search as tools, then explaining its reasoning step by step.

Inspiration

Open a skin app, get a number. Open a style app, get a vibe board. Open a try-on app, get a single overlay. None of them talk to each other — your skin tone never reaches your makeup, your closet never informs your shopping, and a "score" never becomes a next step.

We asked one question: what if an AI agent could wire all of that together — and show its work?

What it does

MirraAI is one guided flow that connects skin health, fashion, and shopping using two captures:

Selfie → skin analysis, skin tone, face attributes, GlowUp recommendations, and face-based VTO (makeup, hair, earrings, necklace).
Full-body photo → clothes virtual try-on inside the Try-On Studio.

From those two images, the agent moves the user through a single decision pipeline:

#	Step	What the agent does
1	Capture	Guided selfie kicks off 3 Perfect Corp analyses in parallel (skin, tone, face attributes).
2	Skin Health	Reasons over 14 concern scores, history, and today's weather; emits insights with callable next-step paths (e.g., /skin/simulate).
3	GlowUp Studio	Plans face-shape + undertone + persona-aware makeup and hairstyle picks — a masculine read gets grooming polish (skin smoothing, defined brows, lip tint) instead of full glam.
4	Smart Closet	Vision-tags every wardrobe item (category, color, formality, occasion) so the agent has structured memory to plan against.
5	Try-On Studio	One unified studio: clothes VTO uses the body photo; makeup, hair, earrings, and necklace VTO uses the selfie.
6	Outfit Builder	Agent loop: closet-first match → identify the gap → Serper product search → VTO validation → Proof Card.
7	Proof Card	A visual receipt before purchase: tone match, style fit, skin safety, what's from your closet ($0), what's new, total spend.

Built on Perfect Corp

The integration is the product. Nine Perfect Corp APIs are exposed as tools the agent picks from:

Skin intelligence (4 APIs)

AI Skin Analysis — 14 concern scores → the health dashboard
AI Skin Tone — undertone + color palette → makeup and fashion logic
AI Face Attributes — face shape + gender → GlowUp planning
AI Skin Simulation — before/after visualization for skin goals

Virtual try-on (5 APIs)

AI Clothes VTO — outfit on the user's body shot
AI Makeup VTO — face-aware beauty application
AI Hairstyle VTO — hair changes with face-shape context
AI Earrings VTO — accessory try-on for complete looks
AI Necklace VTO — jewelry visualization for full-look builds

These nine APIs are stitched together by a custom orchestration layer with Redis caching, so skin tone informs makeup palettes, face shape drives hairstyle picks, closet gaps trigger product search, and every VTO output feeds the Proof Card.

Agentic AI — the part most beauty apps skip

Perfect Corp gives us pixel-accurate analysis and rendering. Gemini is the agent on top — the planner that decides which tools to call, in what order, with what context, and how to explain the result back to the user. Every reasoning step is structured, cached, and visible.

1. Visible reasoning traces, not a black box

Every Gemini call returns a structured steps[] array rendered as a live "AI thinking" trace. The user literally watches the agent work:

Read your face shape as oval. Mapped your coloring to a warm palette. Built a masculine GlowUp plan across makeup, hair, and accessories.

No mystery output — just an explainable decision path.

2. Tool-calling as a first-class concept

The agent picks tools dynamically based on intent. A single skin insight can pull from skin-analysis, weather, and history; a single outfit recommendation can chain closet-match, serper-shopping, clothes-vto, and proof-card-scorer in one decision. The tools the agent actually used are surfaced under each result as Signals, so users (and judges) can audit what was queried.

3. Persona-aware prompt routing

The agent reads gender from Perfect Corp's face attributes and switches both the preset catalog and the system prompt. A masculine face is told "favor grooming polish, avoid bold lipstick / smoky eye"; a feminine face gets the full glam catalog. Same model, different agent persona — driven by upstream signals, not user toggles.

4. Context-aware memory

Skin insights aren't generated in isolation. The agent injects:

Latest 14 skin scores (Perfect Corp)
Trend deltas vs. last scan (Supabase history)
Today's local weather (Open-Meteo)

…so a flat number like moisture = 68 becomes "your moisture dropped 12% this week, low humidity today is making it worse — here are HA serums in your tone." The agent reasons over a mini state, not a single API response.

5. Action handoff — every output is callable

Every recommendation the agent emits comes with a sanitized in-app action path (/skin/simulate, /try-on, /outfit). Tap a recommendation chip, and you're already inside the next tool with the right context loaded. The agent's output isn't a wall of text — it's a clickable next step.

6. Graceful degradation

If Gemini is rate-limited or unreachable, the agent falls back to a deterministic persona-aware planner that produces the same steps, insight, recommendations, and tool_calls_made shape. The UX never goes blank — the agent simply switches reasoners.

7. The closet is the agent's wardrobe memory

Every uploaded item is vision-tagged once (Gemini), then stored as structured metadata. When the agent plans an outfit, it first queries that memory, scores closet candidates against the occasion and weather, and only calls Serper for the gaps it can't fill from your wardrobe. That's why the Proof Card can show "2 pieces from your closet ($0) + 1 new ($48)" instead of a default "buy everything new" list.

How we built it

Frontend — Next.js 16, TypeScript, Tailwind CSS, shipped as a PWA on Vercel
Backend — FastAPI (Python 3.12) on DigitalOcean App Platform
Agent core — Google Gemini 2.5 Flash as the planner/reasoner; structured JSON outputs (steps, insight, recommendations, tool_calls_made) validated with Pydantic
AI / AR tools — 9 Perfect Corp S2S APIs, orchestrated through a custom polling + caching layer
External tools — Serper for live shopping search, Open-Meteo for weather, Supabase for closet + history memory
Data — Supabase (Postgres + Auth + Storage)
Caching — Redis on every Perfect Corp + Gemini call so retries and repeat looks stay instant

Challenges we ran into

Making 9 APIs feel like one agent. Skin scores are numeric arrays, tone analysis returns hex palettes, face attributes return categorical labels — three entirely different schemas, all surfaced in the same dashboard. We built a normalization layer with Redis-backed caching so the agent reasons over a single unified profile, not nine disconnected widgets.

The two-image problem. Skin and beauty need a close-up selfie; clothes VTO needs a full-body shot. We designed the Try-On Studio so it transparently switches between the two captures — the agent picks the right input for each tool, the user never has to think about which image is active.

Turning analysis into action. A score is not advice. We layered Gemini reasoning over Perfect Corp output so "moisture dropped 12% this week" turns into "low humidity today makes that worse — here are HA serums in your skin tone." Reasoning runs against the raw scores, current weather, and prior history.

Keeping the agent honest. Free-text LLM outputs love to hallucinate fake routes ("view_matched_outfits", "shop_for_shoes"). We added a sanitization layer that drops any action that isn't a real in-app path or https:// URL, so the agent's clickable recommendations never 404 in production.

Persona-aware planning. Default beauty catalogs assume a feminine target. We routed prompts and preset catalogs by gender so masculine-presenting users see grooming polish (skin smoothing, defined brows, lip tint) instead of glam looks they would never apply.

Auth at the edge. Our frontend (Vercel) and backend (DigitalOcean) live on different origins. Vercel's rewrite proxy silently stripped Authorization headers on cross-origin redirects — a bug that only surfaced in production. We pivoted to direct CORS-enabled calls, which turned out to be faster too.

What we learned

The biggest insight: users don't want nine AI features — they want one agent that gives them one answer to "does this work for me?" Perfect Corp's API suite covers the full spectrum from skin biology to fashion visualization, which gave the agent a complete toolbox. The hard part wasn't calling APIs — it was designing the agent loop so skin tone informs makeup, face shape drives hair, and closet gaps trigger real product search with VTO previews.

We also learned that proof beats persuasion. When we added the Proof Card — a transparent breakdown of what matches, what's new, and what it costs — test users said it was the feature that made them trust the agent. Showing the math behind a recommendation, plus the explicit list of tools the agent called to reach it, beats any AI-generated compliment.

Finally: showing the agent's work changes how it feels. Rendering steps[] as a live thinking trace, surfacing tool_calls_made as visible "Signals," and giving every recommendation a callable action path turned a generic LLM output into a product users actually trusted.

What's next

Voice-guided agent loop — Hands-free morning walkthrough where the agent narrates its reasoning (Deepgram STT + TTS pipeline scaffolded)
Body-reshape VTO tool — Adds a 10th Perfect Corp tool to the agent's toolbox so clothes drape on different body types, not just overlay on the original
Calendar-aware planning — "You have a client dinner Thursday — here's what works from your closet" (Google Calendar integration started)
Social Proof Cards — Share the visual receipt with friends before buying
Long-term memory — Multi-week skin/style trend reasoning so the agent learns what actually worked for you, not just what scored well