🧵 SWYF — See What You Fit: Project Story

💡 Inspiration

The idea for SWYF (See What You Fit) was born out of frustration — both as a customer and as an observer of how broken the online fashion industry can be.

Every day, over 25% of 131.5M fashion orders are returned — that’s nearly 33 million items daily.
While this may look like a minor inconvenience for big brands, it’s devastating for small retailers and new designers who often operate on razor-thin margins.

When customers return clothes due to poor fit, color mismatch, or inaccurate visuals:

Retailers lose money on reverse logistics, restocking, and damaged inventory.
Many local or small-scale sellers never recover, losing credibility and visibility in a saturated market.
And customers, especially in developing markets, often face fake listings and low-quality products, with no visual feedback before purchase.

This cycle traps both buyers and sellers — customers lose trust, and honest small businesses fade away.

SWYF was built to break that cycle.

🧠 What We Built

SWYF combines AI, AR, and real-time color analysis to let users see how clothes would look and fit before buying — creating a “digital fitting room” that works anywhere.

Key Components

Virtual Try-On Engine: Uses camera input + MediaPipe pose tracking to render lifelike clothing overlays.
Color Analysis System: Matches apparel hues to a user’s unique skin tone and lighting profile.
2D → 3D Garment Generator: Converts flat clothing images into .glb 3D assets using an AI reconstruction pipeline.
Tokenized Reward System: Gamifies shopping with SWYF Tokens for engagement, reviews, and referrals.

🧩 How It Works

Camera Input → Captures real-time video frames.
MediaPipe Processing → Detects hand, face, and pose landmarks.
Coordinate Mapping → Converts 2D keypoints into 3D skeletal data.
Data Transmission → Sends joint updates to Needle Engine.
Needle Engine Rendering → Updates clothing models in WebGL/WebXR.
AR Feedback Loop → Continuously refines visuals as the user moves.

$$ x_{3D} = f(x_{2D}, \text{depth}, \text{scale}) $$

🧱 What We Learned

AI + AR synergy can solve real-world retail problems if optimized for on-device inference.
Designing for trust is as important as designing for accuracy — customers must feel confident before purchasing.
Building a consistent B2B + B2C hybrid model takes architectural foresight (developer API, SDK, and consumer app).
Color accuracy under different lighting is a real challenge — it required extensive testing and normalization.

⚙️ Challenges Faced

Maintaining real-time FPS during AR rendering while running AI inference locally.
Aligning 2D garments onto variable body shapes without distortion.
Integrating a secure token economy without compromising privacy or UX.
Convincing early testers and retailers that virtual try-on isn’t a gimmick — it’s a trust-building tool.

🌍 Impact & Vision

SWYF empowers:

Consumers — to shop smarter, with confidence and fairness.
Small retailers — to compete globally without being crushed by return losses.
Fashion ecosystems — to shift towards personalization, data-driven decisions, and sustainable retail.

Our long-term goal is to make SWYF the Shopify of AR Fashion — accessible, intelligent, and transparent.