Phia Hack

eliminate decision fatigue

Comment

Inspiration

Fashion is deeply personal, yet shopping for it online still feels impersonal — endless scrolling, no context, and zero understanding of who you actually are. We wanted to build a stylist that knows you: your Pinterest boards, your Instagram aesthetic, your budget and occasion. Phia is that stylist — an AI agent that feels less like a search engine and more like a friend who has memorized your entire wardrobe.

What It Does

You describe what you need — "something chic for a rooftop dinner, not too formal, under \$300" — and Phia:

  1. Understands your aesthetic by analyzing your public Pinterest boards and Instagram profile for color palettes, silhouettes, and style signals
  2. Plans a full outfit across every clothing slot (top, bottom, shoes, bag, accessories) using a reasoning LLM
  3. Searches real products across fashion retailers and returns curated, shoppable picks
  4. Groups them into outfits — not just individual items, but complete looks with occasion tags, vibe labels, price totals, and match scores
  5. Lets you virtually try them on — upload a full-body photo and see an AI-generated preview of any piece (or the whole outfit) on yourself
  6. Refines in chat — follow up with "make it more casual" or "add a bag" and the grid updates live

Phia also supports gift mode — style an outfit for someone else by describing them.

How We Built It

Backend — A LangGraph Agent Pipeline

The core is a five-node LangGraph state machine:

user message
    ↓
[Intent Classifier]  ← GPT-4o-mini with structured output
    ↓                       ↓
[Planner]             [Capability Responder]
    ↓
[Executor]  ← parallel async tool calls
    ↓
[Synthesizer]  → 12 recommendations + message
  • The Planner uses GPT-4o (with reasoning_effort) to generate a dependency-aware tool call plan — it decides which clothing slots to fill, whether to fetch Pinterest/Instagram data, and in what order.
  • The Executor resolves dependencies between steps (e.g., analyze_style_images waits for fetch_pinterest to finish) and runs independent steps concurrently with asyncio.
  • The Synthesizer merges all tool outputs into a final ranked recommendation list with a natural-language reply.

Tools include: fetch_pinterest, fetch_instagram_profile, analyze_style_images, search_products (via Hasdata), fetch_style_profile, and get_phia_pricing.

Observability is wired via Langfuse — every LLM call, token count, and node latency is traced.

Frontend — Next.js App Router

Built with Next.js 16 (Turbopack), React 19, and Tailwind CSS v4. Key UI decisions:

  • Masonry columns layout for the outfit grid (no layout library needed)
  • Outfit cards show a piece-image collage (1/2/3/4+ piece layouts)
  • Chat lives in a persistent sidebar on desktop and a bottom sheet on mobile
  • Supabase handles auth (Google OAuth), session/message persistence, and saved items

Virtual Try-On

The try-on route (/api/try-on) accepts a multipart upload of the user's photo + a garment image URL. It converts the person photo to a base64 data URI and passes it directly to the Replicate IDM-VTON model (cuuupid/idm-vton), which generates a photorealistic composite. No cloud storage needed — the data URI goes straight to Replicate's API.

Challenges We Ran Into

Try-on returning HTML instead of JSON. The route was trying to upload to a Supabase storage bucket that didn't exist, causing a redirect to a login page. Fixed by eliminating the storage step entirely and passing base64 data URIs directly to Replicate.

LLM output reliability. The planner must return strict JSON for the agent graph to proceed. We added a retry loop (up to 3 attempts) with markdown stripping and structured output validation.

Async tool dependency resolution. The executor needed to build a runtime dependency graph from step references like $step_1.imageUrls, resolve them in topological order, and batch independent steps for parallelism.

Google Photos Picker API. The OAuth popup flow was always blocked by browsers even with popup permissions granted (the "two-popup problem": blank window + GIS popup). We replaced it entirely with a simple drag-and-drop file upload — zero friction, same result.

Pinterest/Instagram scraping reliability. External scraping APIs are expensive and rate-limited during a hackathon. We built a full mock mode (feature-flagged via config.py) so the entire agent pipeline can run offline with stub data.

Accomplishments We're Proud Of

  • A fully agentic planning loop where the LLM reasons about occasion, season, and formality to autonomously decide which product categories to search — not hard-coded rules
  • Outfit-first UX — most fashion apps return individual items; we group recommendations into complete, styled outfits with names, vibes, and total pricing
  • Virtual try-on for clothing — users can pick any piece from a multi-item outfit and see it on themselves in seconds
  • An end-to-end working product: auth → onboarding → style analysis → LLM agent → product search → rendered outfits → try-on → saved board, all in one session

What We Learned

LangGraph state typing matters. It's powerful for multi-step agent flows, but requires careful TypedDict schemas for every edge — they prevent silent failures downstream.

Replicate accepts data URIs natively. This makes multi-modal pipelines much simpler when you want to avoid managing cloud storage.

Tailwind CSS v4 is a breaking change. The @import "tailwindcss" syntax replaces the old @tailwind directives — utilities and plugins work differently from v3.

Reasoning models add real latency. Extended reasoning (reasoning_effort) is genuinely better at occasion-to-outfit-slot mapping, but adds 2–4 seconds of latency per request — a meaningful tradeoff in a chat interface. If L is total perceived latency and Q is quality improvement, the user experience depends on whether:

$$\frac{\Delta Q}{\Delta L} > \theta_{\text{user}}$$

where $\theta_{\text{user}}$ is the user's patience threshold. For fashion planning (a considered, not impulsive, action), we concluded the quality gain was worth it.

What's Next for Phia Hack

  • Wardrobe integration — let users upload photos of what they already own (scrape from camera roll) so Phia can suggest completing existing outfits rather than buying everything new
  • Fit personalization — 3D outfit rendering instead of 2D
  • Richer social signals — deeper Pinterest board parsing (board names, pin descriptions) and TikTok/YouTube lookbook analysis
  • Price tracking — notify users when a saved item drops in price

Built With

  • apify
  • asyncio
  • brightdata
  • fastapi
  • framer-motion
  • hasdata
  • langchain
  • langfuse
  • langgraph
  • next.js
  • openai-gpt
  • pydantic-settings
  • python
  • react
  • replicate-api
  • replicate-idm-vton
  • supabase
  • tailwind-css
  • turbopack
  • typescript
  • uvicorn
  • vercel
Share this project:

Updates