Projector + Mirror + Camera Set Up.
Calibration Process.
Projection Example.
Self-calibration: a dot mesh projected onto the body to check alignment.
Live tracking (left) and what the projector draws (right), mixing tops and bottoms.
Hands up to add the outfit to your cart. The ring fills on the projection.
Website View.
Cart View.
Our first working loop: a red tee and jeans.

Inspiration

Online try-on apps put clothes on a photo of you. That's not the same as seeing a shirt on your own body, at your size, from the angle you'd check in a fitting room mirror. We wanted that fitting room feeling with nothing to put on and nothing to download. And we wanted checkout to happen with the same kind of gesture you use to browse.

What it does

You stand in front of a projector wearing white. Fitcast finds your body with a camera, warps a real garment image onto your torso and arms, and projects it onto you so it looks like you're wearing it. Swipe your hand to change garments. Raise a hand to switch between shirts and pants. Turn to the side or all the way around, and the projection switches to the side or back of the garment.

Not sure what to try? Say "Magic mirror" and ask for something, like "an interview outfit under 80 dollars." Fitcast's Gemini stylist picks a top and bottoms from the catalog, dresses you in them, tells you why out loud, and puts where to buy them on the wall with a QR code. Find one you like, hold your palm up, and Fitcast checks you out.

How we built it

Pose and masking. MediaPipe pose landmarks and person segmentation on a webcam feed. A one euro filter smooths every keypoint, and a small velocity prediction keeps us ahead of projector lag.
Garment warping. Each garment is split into torso and sleeve regions with hand-marked anchor points. Piecewise affine transforms warp each region onto the matching body keypoints, so sleeves follow the arms independently of the torso. The result is masked to the person, so nothing lands on the wall or your face.
Projector alignment. A calibration tool projects a checkerboard onto a board held at the standing spot, finds it in the camera, and solves a camera-to-projector homography. Every frame goes through that mapping before it's drawn.
Orientation. Shoulder angle plus shoulder depth tells us front, left, back, or right. Hysteresis stops it from flickering mid-turn, and each switch cross-fades.
Gestures. Wrist velocity for swipes. Wrist above the head to switch modes. A held palm fills a projected progress ring for checkout.
Voice stylist. An offline Vosk recognizer listens for the wake phrase, so nothing leaves the laptop until you ask. The request audio goes to Gemini Flash along with the live catalog. Gemini returns one top, one bottom, a one-sentence answer, and shopping picks. The booth speaks the answer, dresses you, and projects the picks and a QR code. Curated retailer links always come first, and if one model is slow the request falls through to the next.
Checkout. The palm hold posts to a Visa sandbox checkout on a background thread, so the render loop never stalls. The confirmation is projected back onto the user.
Stack. Python, OpenCV, MediaPipe, NumPy, SciPy, Gemini, Vosk, one Epson projector, one webcam, and a roll of painter's tape.

Challenges we ran into

Two coordinate systems. The camera and projector see the world differently, and the mapping between them changes with distance. We fixed the standing spot, mounted the camera touching the projector lens, and calibrated once at that depth.
Latency. Capture, inference, rendering, and projector input lag stack up to well over 100 ms. That's enough for the shirt to trail you when you move. Smoothing plus prediction got slow movement looking right.
The projection fights the tracker. The garment lands on the person and changes what the camera sees, which can confuse the pose model. Masking the projection tightly to the body and keeping brightness in check kept tracking stable.
A mirror that hears itself. The booth kept recording its own spoken answers as new requests. We mute the listener while it talks and ignore its own chime.
No side photos. Stores shoot garments from the front and back, never the sides. So we photographed our demo garments ourselves from four angles.

Accomplishments that we're proud of

A projector, a webcam, and a laptop became a working fitting room in a day. The moment the shirt first followed someone's arm as they raised it, the idea stopped being a slide.

What we learned

Projection mapping onto a moving person is mostly a physics problem, not a CV problem. Calibration, lighting, and latency ate more of our time than the model did. We also learned that one garment that tracks perfectly beats ten that drift.

What's next for Fitcast

Depth camera calibration so you can stand anywhere. Retailer catalog ingestion, so any product with front and back photos just works. And a tokenized checkout that remembers returning shoppers, so the second purchase takes one gesture.