Tillie

Stop doom-scrolling for that one jacket. Tap the T and let Tillie decide.

Comment

Inspiration

Every outfit I've bought this year started on TikTok. I'd see a fit I loved, screenshot it, then spend twenty minutes reverse-searching, opening five store tabs and guessing my size at brands I'd never tried. Half the time the pieces were from unknown brands, so I never found them at all. Growing up in Indonesia made it worse, because shopping from US and European stores meant surprise currency conversions, duties and card fees at checkout.

We wanted the moment of "I want that" to turn straight into "it's on the way," without the tabs, the guesswork or the surprise fees.

What it does

Tillie is a GenAI shopping companion that lives on top of your feed.

  1. Tap the T on any video. A floating Tillie bubble sits over a TikTok-style feed. One tap scans the look and lists every piece: jacket, top, jeans, shoes and accessories.
  2. Compare real matches. For each piece, Tillie shows the exact match first, then cheaper look-alikes across Zara, Uniqlo, Aritzia, Cider, Princess Polly and more, already in your size.
  3. Fit memory. Tillie learns your size from what you've bought before and your measurements, then translates it across brands (an XS at Zara might be an S at Aritzia).
  4. Closet that sorts itself. Saved pieces auto-file into collections like Summer tops, Cozy knits and Jeans, and land on your avatar right away.
  5. 3D fitting room. A spinnable 3D body built from your measurements. Pieces snap into inner, mid and outer layers, so a jacket sits over a hoodie over a tee. You can shuffle, lock favorites and save full outfits.
  6. See it on me. Pick a full-body photo and Gemini's image model puts you in the outfit you styled.
  7. One global checkout. Buy a whole outfit from several stores in different countries with one Visa payment. Currency, duties and international card fees are shown up front, then an SMS code verifies the payment and each store gets its own tracking ID.
  8. Rewards and community. Every dollar spent, look scanned or outfit posted earns points that unlock real brand perks, like 10% off at Zara. You can post outfits, follow friends and try their looks in one tap.

How we built it

  • App: React Native with Expo (SDK 57), so it runs on a real phone through Expo Go.
  • 3D fitting room: raw three.js rendered on expo-gl. The body and every garment are built from lathe geometry and scaled from the user's measurements (bust, waist, height), with layer offsets so outer pieces sit on top. There's a 2D SVG fallback for devices without GL.
  • Gemini API: our Node/Express server sends screenshots to Gemini with a structured prompt and gets back each clothing item as JSON (category, name, hex color, search phrase). We rank products by category and color distance to find the closest matches. A second route sends the user's photo and their styled outfit to Gemini's image model for virtual try-on.
  • MongoDB Atlas: every order is saved to Atlas, and each purchase writes to a fit_memory collection that teaches Tillie your size at each brand.
  • Design: a Y2K "digital wardrobe" look with retro window frames, a pixel font, script accents and a pink and periwinkle palette, designed in Figma and Canva and rebuilt in code.
  • Quality and security: we ran VEYZL's automated audit on our pull requests. API keys only live on the server, never in the app.

Challenges we ran into

  • Making 3D feel like 3D. Our first avatar was a flat SVG that just squashed when it turned. We rebuilt it in three.js, then fought boxy shoulders, helmet-shaped hair and legs poking through skirts until the layers sat right.
  • Real clothes are hard to drape. Physically simulated fabric wasn't realistic in a weekend, which is why we added Gemini image try-on on a real photo.
  • Hackathon networking. The venue Wi-Fi blocked our phone from reaching our laptop server, so we tunneled the server through Cloudflare to keep demoing.
  • Free-tier limits. Gemini's image model has no free quota, so try-on needs a paid key, while outfit detection runs on the free tier.

Accomplishments that we're proud of

  • Going from a hand-drawn sketch to a working app on a real phone in one weekend.
  • Being able to incorporate cross-continents payments because this was always a personal struggle for us when it comes to online shopping.
  • A full loop: see an outfit, identify every piece, style it on your own 3D body, and check out across countries in one payment.
  • Fit memory that actually translates sizes between brands, which solves our own biggest shopping frustration.
  • Figuring out the most efficient flow of the app (what interface is best for consumer experience)
  • An interface that looks like us. It feels like a digital closet, not a generic shopping app.

What we learned

  • How to use multimodal AI as a real feature, not a gimmick: structured prompts and JSON output turn Gemini into a reliable part of the pipeline.
  • How 3D rendering works under the hood (geometry, lighting, camera, layering) without a game engine.
  • That trust is part of the product. Showing fees up front, labeling AI previews and never storing photos matter as much as the features.
  • The design and learning curve is as important as the technical works, because it draws people into actually using your product

What's next for Tillie

  • Real product APIs and affiliate partnerships, so matches are live inventory and Tillie earns a commission on every outfit.
  • Brand partnerships for rewards, turning points into real discounts and fewer returns, since shoppers already know their size.
  • Better try-on with product photos fed to Gemini for more accurate fabric and fit.
  • Deploying the server on Vultr with our own domain, so Tillie works anywhere.
  • ACTUAL USAGE: we want to see our friends and family benefit from this app in the future!

Built With

Share this project:

Updates

Submission history