Onboarding page
My Wardrobe Page
Wardrobe try showing You Corp Results
Wardrobe try on showing Crusoe Nemotronic review
Sponsored Page showing sponsored results and your wardrobe matches by Nemotronic suggestions
Try on a Sponsored (by Nemotronic) result. You Corp renders the image.
Saved favorite styles into your "Lookbook"
Profile Page

Inspiration

Whenever I need to go for an event, its a hassle trying to find the right combination of things to wear. With an age where style matters, its becomes critical to find the style that you are comfortable with and that suits the occasion. Also with growing number of clothes we buy online, and the number of clothes we own we always ask these questions:

"Will I look good in this?"
"Does it match my favorite pair of jeans?"
"Should I buy this? Do I already have something like this?"

Additionally with growing number of retailers owning some amazing clothes, its always an hassle to go to each retailer's website and try to find something that goes well with the wardrobe.

This inspired us to use AI to solve this gap, creating a fun multi-modal project.

What it does

Short summary on What it does

Clothink solves exactly these problems. It is an AI wardrobe that helps you try on clothes virtually, save your favorite looks in a lookbook, and also tells gives you a style score for each combination you try on. For each new garment you add to your wardrobe, it maintains a profile based on AI reasoning for the newly added garment that depicts your entire wardrobe. Also, our App also has a way where external companies can show Ads (sponsored Ads) about their products such that Advertisement is in tandem to what the User actually needs. This way both the User and the Ad provider wins.

Detailed list of what we implemented for the hackathon:

Virtual try-on for any garment. Upload a clothing photo and you see yourself wearing it. Works for tops, bottoms, dresses, outerwear, hats, scarves, bags, and shoes.
Your own wardrobe online. Upload your clothes, filter by category, color, and favorites. Mix & Match lets you render any combination of your own pieces on yourself.
An AI stylist that pushes back. Every try-on runs a 7-step agent on NVIDIA Nemotron. It scores the pairing on color, formality, occasion, and silhouette, checks for duplicates you already own, picks the best combinations from your closet, and writes an honest take — including "skip it, you already own two like this."
A profile that updates itself. Every time you add a wardrobe item, a background classifier tags its style and rolls up your aggregate taste into a radar chart across 8 dimensions.
Brand catalog with recommendations. Retailers can drop items into a central catalog. We embed each one with Jina CLIP and use pgvector plus a Nemotron rerank to find catalog pieces that pair with what you already own.
Lookbook. Save the renders you like. Tap one and we use it as the base for a new try-on, so you can keep building on outfits you already love.

How we built it

Big BIG thanks to the Sponsors Crusoe and Perfect Corp:

1. Perfect Corp — for the virtual try-on rendering. We use five task endpoints, all async. Each POST returns a task_id and we poll the GET endpoint every 3 seconds until it comes back success or error, with a 4-minute timeout per task.

POST /task/cloth-v3 — body garments: tops, shirts, blouses, dresses, jeans, skirts, shorts, outerwear, jackets, blazers. Returns a render URL plus a dst_id we can reuse for chaining.
POST /task/hat — 5 styles: sporty_casual, urban_fashion, vacation_casual, warm_cozy, bohemian.
POST /task/scarf — 5 styles: french_elegance, light_luxury, cottagecore, modern_chic, bohemian.
POST /task/bag — 4 styles: parisian_chic, urban_chic, mediterranean_chic, art_deco_style.
POST /task/shoes — 5 styles: minimalist, bohemian, cottagecore, french_elegance, retro_fashion.

Each task takes a base image (the user's profile photo, or the output of a previous render) and a garment image, then produces a new render. To put multiple items on the user we chain calls — the URL from one render becomes the source for the next. A 4-item outfit means 4 sequential Perfect Corp calls.

The ordering rule was the trickiest part. The accessory endpoints regenerate the entire outfit during their pass, so if you render them after a body garment, the real clothes underneath get repainted into something the model made up. We always render accessories first and cloth-v3 last. That way the clean garment swap is the final pass and the user's actual top, dress, or pants ends up sitting on top of whatever the accessory render produced. Within each group we keep the new item last so it gets the freshest, topmost layer.

For the same reason, we skip the solo accessory render in the new-item try-on flow. If the user is trying on a single hat, calling /task/hat alone would replace their whole outfit with a hallucinated one. Instead we route them straight to the closet picker so they can chain the hat with one of their own body garments via cloth-v3 and get a clean, trustworthy composite.

Two more things we had to work around:

Perfect Corp only accepts direct image URLs (.jpg/.png), not gallery pages or raw bytes. We mirror every external image through Cloudinary (services/cloudinary.js → ensureCloudinaryUrl) before sending it — catalog product photos, saved look renders being used as a new base, and any wardrobe item being chained into a composite.
Category routing is handled by an internal map in services/perfectCorp.js. "Purse", "tote", "backpack", and "clutch" all route to /task/bag; "sneakers", "boots", "heels", "loafers", "sandals" all route to /task/shoes; "beanie" and "cap" both route to /task/hat. Body garments fall through to cloth-v3 with a garment_category of upper_body, lower_body, full_body, or auto.

2. Crusoe Nemotronic (the biggest differentiator for this Apps competition in the market): NVIDIA Nemotron-3-Nano-Omni Reasoning 30B. We use it across four agentic workflows:

The 7-step stylist agent for try-ons (services/stylist/orchestrator.js). Every time the user tries on a new garment, this pipeline runs. Three of the seven steps are Nemotron vision/reasoning calls: an Analyzer that identifies the item and assesses how it fits the user, a Stylist + Critic that reranks candidate outfits from the user's wardrobe with a 0-10 rubric on color, formality, occasion, and silhouette, and an Assessment writer that composes the final honest verdict. The other four steps are deterministic JS tools (duplicate check, candidate generator, versatility scorer, gap analyzer) so Nemotron only does the parts an LLM is actually good at. Each step pushes a human-readable line into a trace[] the UI exposes in the "How the stylist thought about this" panel. The same pipeline also runs from the Recommendations page when you tap "Try It On" on a catalog item.
The Mix & Match wardrobe agent (services/stylist/wardrobeOrchestrator.js). When you pick a few of your own clothes to render together (no new item), Nemotron does its own multi-step pass in parallel with the Perfect Corp render. It analyzes the outfit as a whole, gives it a name and occasion, scores it on the same rubric used for new-item try-ons, calls out any coverage gaps, and writes a critique.
The style classifier (services/styleProfile.js). Every wardrobe add fires a background Nemotron vision call that tags the item with one of 8 style labels: casual, formal, minimalist, bohemian, streetwear, romantic, sporty, classic. Those tags roll up into the radar chart on the Profile page so the user's aggregate taste profile updates by itself.
The recommendation reranker (services/llm.js → rankCandidates). The Recommendations page uses pgvector cosine similarity to pull the top 15 catalog matches per wardrobe item. Nemotron then picks the best pairing per slot and writes a one-sentence style reason for why it works.

Every Nemotron call is Crusoe-first with an automatic Groq llama-4-scout fallback, so one bad provider call never breaks the session.

Now coming to the tech specs:

Frontend: React 19, Vite 8, React Router 7, Tailwind 4
Backend: Node.js, Express 5
Database: PostgreSQL with pgvector + pgcrypto
Image hosting: Cloudinary (we mirror every external image so Perfect Corp can actually fetch it)
Virtual try-on: Perfect Corp YouCam, AI Clothes V3 plus the hat, scarf, bag, and shoes endpoints
Stylist LLM: NVIDIA Nemotron-3-Nano-Omni Reasoning 30B via Crusoe
Image embeddings: Jina CLIP v2, 1024-dim, stored in pgvector

The whole thing is one Express server and one React SPA. All API keys are server-side. The interesting code is in three places: the multi-step stylist agent in services/stylist/, the Perfect Corp chained render logic in services/perfectCorp.js, and the recommendation query in routes/recommendations.js.

Challenges we ran into

Perfect Corp's API is async. You POST a task, get back a task_id, and have to poll until it's done. Each garment took 15-25 seconds and a multi-item composite has to be chained call by call. We poll for up to 4 minutes per task and run outfit combos in parallel so the user mostly waits for rendering, not for anything else we're doing.

The accessory endpoints (hat, scarf, bag, shoes) caused us a headache. They regenerate the whole outfit, not just the accessory. So if you render shoes after a top, the real top gets repainted into something the model made up. We fixed it by always rendering accessories first and body garments last, since cloth-v3 is a clean swap that only edits its garment region.

Perfect Corp also needs direct image URLs (.jpg/.png), not gallery pages or raw bytes. We mirror every external image through Cloudinary before sending it.

Nemotron's reasoning trace ate our token budget more than once. The model emits its chain-of-thought before the JSON answer, so if max_tokens is too tight the answer comes back null and we silently fall back to Groq. We ended up with a tiered buffer — 16K for text-only, 32K for multimodal — and one retry at 48K if the first call returned nothing.

Our style classifier was quietly breaking the profile radar. The classifier maps each item to one of 8 style tags. When the model returned tags outside that vocab, the normalizer stripped them all, the function bailed early, and the profile aggregate never updated. We added a log line for that path so it's not invisible anymore.

The recommender was a fight too. Pure vector similarity pulled visually similar items, which meant the top results were almost always the same color and category as the wardrobe item being matched. We had to encode the business logic in SQL — gender filter, no jeans-pair-with-jeans, treat dresses as a full outfit, no same-color matches — and then let Nemotron rerank the top 15 per slot.

Accomplishments that we're proud of

We fed the sponsored catalog data in manually for the hackathon and it still works end-to-end.

The reasoning trace from the stylist agent is something you can actually open in the UI. There's a panel that shows each of the 7 steps and the score breakdown, so judges can verify that the rubric fired and the verdict lines up with the signals.

The honest assessment is one of our favorite parts. The scoring rubric is anchored — the default for an average pairing is 5 or 6 out of 10, not 8 — with a weakest-link rule that caps the overall score if any axis is bad. The verdict text mirrors that. The app is allowed to say "skip it" and it does.

We also got the LLM resilience right. Every call hits Crusoe first and falls back to Groq llama-4-scout if anything goes wrong. One bad call doesn't break the user's session.

The Perfect Corp chain works across every accessory endpoint, not just the standard cloth one. We chain cloth-v3 + hat + scarf + bag + shoes with a deterministic order so a 4-item composite renders cleanly.

The Bought-it loop is the part we're most proud of conceptually. When you buy something off your wishlist, it becomes a wardrobe item, which means it shows up in future try-on pairings and recommendations. The wardrobe gets smarter the more you actually use the app.

What we learned

The hardest part wasn't any single API. It was the choreography — running the stylist agent and the Perfect Corp render in parallel, surfacing partial state while you wait, and falling back cleanly when something fails. Those decisions made more difference than picking faster models.

Prompt engineering is mostly anchoring. Telling a model to "be honest" does nothing. Telling it "6 is workable but unremarkable, 4 is same-tone repetition like blue + blue, 2 is pieces actively fighting" gets you verdicts that actually push back.

pgvector turned what would have been a Python pipeline into a single SQL query. The whole recommender — vector recall and business rules — fits in one JOIN LATERAL.

Showing the model's reasoning to the user made the same agent feel twice as smart. Most apps hide it. We made it expandable.

Multimodal models reason longer when you give them more images, and the token budget needs to scale with the input or your answer gets clipped. A tiered buffer with one retry handled it for us.

What's next for Clothink a Closet that thinks back

We would like to fine tune our stylist agent ourselves to provide better results. We think this can be done by providing it real world examples of fashion pictures that highlight color combinations.

Improve latencies for our system by using vector values instead of complete images.

OCR for price tags so you don't have to type the price.
A native mobile app. The current UX is mobile web but a native iOS app would unlock the in-store moment.
Multi-user accounts. Auth is already in the schema; the demo runs on a single user. Wiring it up is a day of work.
Direct retailer feeds (Walmart, Target, H&M, Zara) instead of the manual catalog seed.
A sustainability score so the honest take can factor in CO₂ and re-wear potential.
Social sharing. Share a look, let friends vote before you buy. Choreograph you lookbook like instagram that you can share to social media!!
Makeup try-on. Perfect Corp supports it; we'd unlock it on a second screen alongside garments.
Closet analytics for brands. Anonymized "what gets paired with what" data as a B2B revenue layer on top of the catalog placements.

Try it out

Note- images take 2 minutes to render due as we call You Corp with chaining different garments one after another. We plan to discuss with YouCorp POCs and see if we can optimize our process.

You can try out the project by clicking our hosted link here: https://closet-b0ug.onrender.com/