StyleSense

Upload a selfie, add clothes from any URL, see yourself wearing them instantly. AI try-on, event scenes, runway videos, and a voice stylist that knows your wardrobe.

Comment

Inspiration

Online clothing returns cost the industry $816 billion per year globally — almost entirely because shoppers can't visualize how clothes will look on their body. Every "buy, try, return" cycle wastes packaging, logistics, and time.

We wanted to flip that. What if you could upload a selfie, paste a product URL, and see yourself wearing the item — before buying? That was the spark: a personal AI wardrobe that makes every shopper their own model.

What it does

StyleSense is an AI-powered virtual wardrobe and try-on web app:

  1. Upload a selfie → a full-body editorial 3D avatar is generated from your photo
  2. Add clothes from any product URL (Amazon, Zara, etc.) or by uploading images — Claude vision auto-detects multiple items in a single photo
  3. Virtual try-on — see yourself wearing any item or outfit combination, powered by Runway gen4_image
  4. Event scenes — place yourself at a "beach wedding", "Tokyo street", or any scene
  5. Runway walk — animate any look into a 5-second video with gen4.5
  6. AI Stylist — voice avatar (Aria) powered by Runway's Characters API; she knows your entire wardrobe via synced knowledge base and gives personalized picks in real time
  7. Social — save outfits, share try-ons with friends, real-time chat

How we built it

Frontend: Next.js 14 App Router + TypeScript + Tailwind CSS + GSAP + Framer Motion. All pages are server-side auth-guarded via Supabase middleware.

Backend: FastAPI (Python 3.12) with 50+ REST endpoints organized by feature (avatar, tryon, wardrobe, stylist, friends, chat). Auth enforced via JWT dependency injection on every route.

Databases:

  • Amazon Aurora PostgreSQL Serverless v2 (ap-south-1) — primary data store for users, wardrobe items, try-on results, and outfits. Connected via SQLAlchemy with IAM-based token auth (no static passwords — boto3 mints short-lived credentials at runtime).
  • Supabase — Auth (JWT issuance + verification), Storage (public HTTPS image hosting for Runway), and Realtime (social: profiles, friendships, messages, chat threads).

AI stack:

  • Runway gen4_image / gen4_image_turbo — garment isolation, try-on compositing, event scene placement
  • Runway gen4.5 — image-to-video animation (5s ramp walks)
  • Runway Characters API — WebRTC voice avatar with dynamic personality and wardrobe knowledge
  • Anthropic claude-haiku-4-5 — stylist chat, outfit suggestions, multi-item detection from images (vision)
  • LangGraph — agentic styling workflows
  • rembg (ONNX) — local garment segmentation fallback

Challenges we ran into

  • Runway image URL requirement: All images must be public HTTPS — no localhost. Every selfie and garment must be re-hosted to Supabase Storage before being passed to any Runway API call.
  • Aurora IAM auth in Python: Wiring boto3 token minting into SQLAlchemy's connection pool required a custom creator function that refreshes the token before each connection.
  • Multi-item garment detection: A single wardrobe photo often contains 5+ items. We used Claude vision to detect, crop, and isolate each item before the Runway garment cleaner runs — saving credits vs. processing the full image.
  • Voice avatar knowledge sync: Runway's Characters API uses a persistent knowledge base. We solved real-time wardrobe awareness by PATCHing the shared stylist character's personality field at every voice session start, embedding the user's current wardrobe inline.
  • Credit budget: 50,000 Runway credits total. gen4.5 video costs 60–100 credits per 5s. We used gen4_image_turbo (2cr) during dev and reserved gen4_image (5cr) + gen4.5 (60cr) for demo quality.

Accomplishments we're proud of

  • Full-stack AI pipeline from URL paste → cleaned garment → try-on → event scene → animated video, all in one flow
  • Voice AI stylist that actually knows what's in your wardrobe in real time
  • Aurora + IAM auth wired cleanly into a FastAPI async app
  • Multi-item wardrobe detection (one photo → 5 garment rows) saving 80% of manual entry time
  • The compare slider: drag to reveal before/after is instant "wow moment" for any judge

What we learned

  • Runway's Characters API is surprisingly powerful for building real-time voice experiences — the knowledge base + WebRTC combination is production-grade
  • Aurora Serverless v2 is excellent for hackathon use: near-zero config, auto-scales to zero, IAM auth avoids secret management complexity
  • Garment isolation quality is the single biggest lever for try-on realism — good segmentation → good compositing

What's next for StyleSense

  • Browser extension to try on any item from any retailer in one click
  • Body measurements via pose estimation for size recommendations
  • Outfit occasion scoring ("is this appropriate for a job interview?")
  • Group style sessions — try on the same outfit as your friend simultaneously

Built With

  • anthropic
  • aurora
  • fastapi
  • langgraph
  • next.js
  • runway
  • supabase
Share this project:

Updates