Capsule Drop

Your closet, optimized: search by text or images, auto‑plan 7 outfits, and export a lookbook — powered by TiDB vector search.

Comment

Capsule Drop — Hackathon Submission

Inspiration

  • We wanted a stylist that works with your real closet, not just shopping links.
  • Search results often collapse to one product or repeat variants. We set out to fix consistency and diversity using product‑level deduplication and better retrieval.
  • Fast “from inspiration to outfit”: upload images or type a vibe, then get a week of balanced outfits and a shareable lookbook.

What it does

  • AI‑assisted product search with both text and image embeddings.
  • Image recommendations: drag & drop images; get visually similar products.
  • One‑click weekly capsule (7 outfits) with category balance (top/bottom/shoes or dress+shoes), budget awareness, and brand diversity.
  • Gap Finder: quick buys to unlock more outfit combos.
  • Export: PDF lookbook right from the browser.

How we built it

  • Backend: FastAPI (Python), TiDB Serverless (MySQL‑compatible) with vector search.
  • Embeddings: OpenCLIP ViT‑B/32 for text and images; stored in catalog.embedding and catalog.img_embedding.
  • Vector retrieval: VEC_COSINE_DISTANCE + HNSW vector indexes. Auto mode ranks by the better of text/image distances with NULL guards.
  • Product‑level dedupe: composite key (ASIN → brand+canonical title → URL → id) to avoid repeated variants while preserving variety.
  • Logging: structured logs for SQL, timings, and uniqueness metrics (uniq_prod_pre/uniq_prod_post).
  • Frontend: React + Vite + TypeScript + Tailwind. Visible “Dedupe same product” toggle; client can over‑fetch and dedupe.
  • Planner: category‑aware retrieval + greedy/ILP stubs to assemble a week with per‑event budgets and brand caps.

Challenges we ran into

  • Search collapsing to a single product despite a large catalog. Root cause: over‑aggressive ASIN dedupe and limited over‑fetch.
  • Aligning text vs image retrieval so both work under one ranking strategy.
  • Canonicalizing titles across retailers (stop‑words, color terms, minor punctuation) to reliably merge near‑duplicates.
  • Balancing variety vs relevance in the capsule planner without over‑engineering the solver.

Accomplishments that we're proud of

  • Consistent, varied search and image recommendations with product‑level dedupe.
  • Clear diagnostics: logs show raw vs post‑dedupe counts and top result IDs with distances.
  • Smooth UX: image uploads, Similar items, Filters, and a one‑click PDF export.
  • Extensible planner that enforces per‑event category balance and weekly brand caps.

What we learned

  • Product‑level dedupe needs multiple signals (ASIN, brand+title, URL) to be robust across marketplaces.
  • Over‑fetching before dedupe dramatically improves variety without sacrificing precision.
  • Small logging improvements accelerate debugging more than expected.
  • Combining text and image signals benefits both retrieval and planning.

What's next for Capsule Drop

  • Stronger planner: finalize ILP/hybrid with color/occasion/palette constraints and no‑repeat rules.
  • Better URL canonicalization per retailer; optional colorway‑aware dedupe (treat color variants as distinct when desired).
  • Persist outfits and share links; add Reel export.
  • Performance: vector index on image embeddings everywhere and caching hot queries.
  • More controls in UI: brand cap, cooldown days, and per‑category knobs.

Tech highlights

  • TiDB Serverless Vector (HNSW over VECTOR(512)), MySQL‑compatible SQL with VEC_COSINE_DISTANCE.
  • OpenCLIP ViT‑B/32 for embeddings, runs on Apple Silicon mps when available.
  • FastAPI + React (Vite) + Tailwind; in‑browser PDF via html2canvas + jsPDF.

Demo flow

1) Type a query or upload inspiration images. 2) Toggle “Dedupe same product” as needed for variety vs uniqueness. 3) Explore Similar items or run the Capsule planner for a 7‑day board. 4) Export the lookbook as a PDF.

Built With

Share this project:

Updates