Halloween.Supply

Turn any screenshot into a unique costume you can buy, then try it on live. OpenCV 5 on AWS Lambda isolates the outfit, pulls its colors and tracks your pose so the look fits your body.

Comment

Inspiration

Most Halloween costumes start with a picture: a character from a show, a look someone wore on social media, or a still from a film. The usual way to shop is to describe that picture in a search box and dig through bagged polyester costumes that look nothing like it. We wanted to start from the picture itself and finish with real clothes from real shops, and let you see them on your body before you pay.

What it does

Halloween.Supply turns a screenshot into a costume you can buy, then lets you try it on.

  1. Screenshot to costume. Upload any image of a character or outfit. Within 30 seconds, the app breaks it into the garments you'd buy, searches both Amazon.com and independent retailers for each piece, and streams matches onto a board as they arrive. Each match has its photo, price and sizes.
  2. Photo try-on. Upload or take a photo, and the app renders you wearing the pieces you selected in a clean studio shot.
  3. Live mirror. Turn on your camera (or pair your phone with a QR code) and see yourself in the costume as live video. You change the look by voice: "keep the hat, choose the cheapest jacket", "I already own the boots", or "try a cute vampire look".
  4. Checkout. Pieces from several merchants go into one cart. At checkout, the app checks each merchant's current price, stock and size again, and asks you to approve any price change.

There's also a catalog of 25 complete costumes, each with $, $$ and $$$ options for every piece, so you can start from one of those instead.

How we built it

OpenCV 5 does the image analysis between the camera and the models.†

  • On a screenshot, OpenCV's DNN module runs person segmentation and pose estimation, so the app isolates the foreground character from the background. It also crops each garment region (head, torso, legs, feet) from the pose keypoints. Each piece's search then gets its own crop and a dominant-color palette from k-means in Lab color space, which gives better input than the full screenshot.
  • When we compare a merchant's product photo with its garment crop, color-histogram distance gives a fast first check before the slower vision-model review.
  • For try-on, OpenCV checks each photo and live keyframe before any generation runs. It measures blur (Laplacian variance), exposure and full-body framing from the pose keypoints, so the app can say "step back so your feet are in frame" instead of spending a render on a bad input.

AWS runs the OpenCV pipeline.† The analysis service is a container image with OpenCV 5 on AWS Lambda behind API Gateway. The Next.js app on Vercel calls it for screenshot analysis and try-on frame checks. Images are processed in memory and not stored.

The rest of the stack:

  • gpt-6-luna reads the segmented screenshot into a structured shopping list and reviews product photos against the reference.
  • Firecrawl and live web search find merchant pages, and we read price, stock and size data directly from merchant HTML and Shopify endpoints.
  • Discovery and enrichment are separate steps, so likely matches stream to the browser as NDJSON immediately, and facts fill in afterwards.
  • gpt-image-2.5 renders photo try-ons. Decart's Lucy 2.5 realtime model renders live video stream. GPT-Live handles voice.
  • OpenAI's Decisions API chooses only among product IDs we supply, so voice commands can't invent a product or a price.
  • The phone camera connects over WebRTC, with Turso for short-lived signaling and a TURN relay for networks that block direct connections.
  • Various LLMs served via AWS power the personal shopper chat and semantic search.
  • Catalog data lives in Turso, and payment runs through Stripe Checkout.

Challenges we ran into

Speed and accuracy pulled against each other in screenshot sourcing. Our first version checked every merchant fact before it showed a match, so the board waited for the slowest website. We split discovery from verification, set a hard 30-second budget, and kept partial results when time ran out.

Merchant data is messy. Prices are missing, sites report the wrong currency, and listings call the same horned helmet a "Viking hat" or a "devil headband". We decided never to fill a gap with a guess: an unknown price stays unknown, and an item without a confirmed price and stock status can't go into the cart.

Live video had practical limits. A blurry or badly framed camera input wastes an expensive render. OpenCV's helped solve this problem.

Accomplishments that we're proud of

  • A random screenshot becomes a board of real, purchasable pieces in under 30 seconds.
  • You can change a live-video costume by talking to it, and every change maps to a real product with a real price.
  • Our end-to-end checks call the real merchants, models, database and Stripe, without mocks, so a passing check means the actual flow works.

What we learned

Showing likely matches first and refining them later felt much faster than showing nothing until every match was checked, even when the total time was similar. Cheap, deterministic computer vision before the expensive models improved the results and cut cost. Constraining generative models to choose from real options, instead of producing answers freely, made it safe to let them act on someone's cart.

What's next for Halloween.Supply

  • Size suggestions from body proportions measured on the try-on photo, so shoppers don't have to guess between M and L.
  • Pose-anchored overlays from OpenCV that keep the live costume steady during fast movement.
  • Group costumes, where one screenshot of a cast becomes coordinated carts for each person.
  • Automated fulfillment, to replace today's manual merchant orders.

Built With

Share this project:

Updates

Submission history