Visual Hobby Market

Turn your Pinterest inspiration into hobbies you can start. Claude Vision reads your images; Qloo Taste AI finds matches across music, film, books, and places.

Comment

Inspiration

We all save far more inspiration than we ever act on. Pinterest boards, screenshots, and outfit photos pile up, but they rarely turn into anything we actually do. We wanted to close that gap: take the visual taste behind a photo and turn it into a hobby you can start this week.

The name is a nod to the "market" feeling of picking trinkets. You don't browse a generic list. You choose the pieces that fit you.

What it does

Users hang 1 to 4 photos on a clothesline. The app extracts Color DNA swatches, and users can add aesthetic tags such as botanical, lo-fi jazz, or dark academia.

When they click "Find My Hobbies," the app:

  1. Sends the images to Claude Vision, which returns a structured reading of the lighting, textures, and mood, plus custom hobbies with starter kits.
  2. Converts that reading into seeds for Qloo Taste AI: tag queries and anchor entities.
  3. Queries Qloo for cross-domain matches in music, film, books, and places.

The result is a persona, a set of hobby cards with match scores and starter kits, and a Qloo Mixtape of recommendations across four domains.

How we built it

  • Frontend: Next.js 14, Tailwind CSS, and a Y2K-inspired interface.
  • Backend: Next.js API routes run the full pipeline. A Python FastAPI service and CLI mirror the same logic.
  • Vision: The Anthropic Messages API with claude-haiku-5-5. We force a tool_use call named visual_signals, so the output is a schema-validated object rather than free text.
  • Qloo: A two-step flow. Seeds are resolved through /v2/tags and /search in parallel, then /v2/insights runs once per domain in parallel.
  • Testing: Tests cover the successful vision path and the fallback path, including a check that no fixed visual tags are injected when vision fails.
  • Deployment: Vercel.

Because the Qloo calls run in parallel, total latency is bounded by the slowest step in each stage rather than the sum of all calls:

$$T_{\text{total}} \approx T_{\text{vision}} + \max\left(T_{\text{tags}}, T_{\text{search}}\right) + \max_{d \in {\text{artist, movie, book, place}}} T_{\text{insights},d}$$

Challenges we ran into

  • Thinking tokens ate the output budget. Haiku enables extended thinking by default, and in early runs it consumed all of max_tokens before emitting the tool call. We disabled thinking explicitly and set max_tokens to 2500.
  • Serverless timeouts. The default Vercel limit is too short for vision plus Qloo calls. We raised the function duration and set explicit timeouts: 25 seconds for vision with one retry, and 5 to 10 seconds for each Qloo request.
  • Image size. Phone photos are large. We compress them client-side to 768px and resize again on the server before sending them to the API.
  • Failing honestly. When vision or Qloo fails, the app says so. It falls back to neutral tags, and judge mode shows which source produced each signal. We preferred a visible fallback to a convincing fake.
  • Two pipelines to keep in sync. The web app and the CLI implement the same logic, so changes in one can drift from the other.

What we learned

  • Forced tool use gives far more reliable structured output than parsing free text, which matters when the output feeds another API.
  • Multimodal input is the bridge that makes a taste graph usable. Qloo works with entities and tags, and people think in images.
  • Transparency is a feature. Showing each API call and its source builds trust, especially in a live demo.
  • Design and engineering are one system here. The decision to reveal all results at once depended on latency, and latency depended on how we structured the parallel calls.

What's next

  • Persist user sessions and saved hobby boards.
  • Add a community space once it can scale without crowding the main experience.
  • Link starter kits to real local shops.

Built With

  • claude-api
  • claude-vision
  • fastapi
  • multimodal-ai
  • next.js
  • python
  • qloo-api
  • qloo-taste-ai
  • react
  • tailwind-css
  • tool-use
  • typescript
  • vercel
Share this project:

Updates

Submission history