Hidden Gem

Find the next Hamilton before it's Hamilton. Hidden Gem reads the signals across NYC's theatre pipeline and flags breakout shows while tickets are still $25!

Comment

Inspiration

Every blockbuster Broadway show — Hamilton, Hadestown, A Strange Loop — started Off-Broadway, surrounded by signals that were obvious in hindsight: OBIE wins, sold-out runs at pipeline venues like The Public or Playwrights Horizons, Jonathan Larson grants. But nobody was systematically watching for those signals in real time. We're theatre fans in NYC, and we kept asking ourselves: what if an AI agent could read the patterns that precede a breakout — before the rest of the world catches on? That question became Hidden Gem.

What it does

Hidden Gem is a predictive AI agent that identifies Off-Broadway shows poised to transfer to Broadway. It scans live web data across five signal categories — venue calendars, awards (OBIE, Lortel, Drama Desk), grants (Jonathan Larson, Sundance Theatre Lab), critical heat, and production history — then ranks current Off-Broadway shows by how closely they match the pre-breakout fingerprints of past hits. The result: a ranked signal board backed by evidence, and a top pick delivered with a theatrical reveal.

How we built it

  • Orchestrator agent powered by Claude Sonnet with tool use, running an autonomous loop that scrapes, analyzes, and stores signals
  • Nimble API for live SERP-based web intelligence — the agent calls it to search venue calendars, award nominations, and grant recipients in real time
  • ClickHouse as our analytics warehouse, storing signals with source URLs so every prediction has a verifiable evidence trail
  • FastAPI backend with Server-Sent Events for real-time streaming of agent tool calls, signal storage, and the final brief
  • Vanilla HTML/CSS/JS frontend with a two-panel dashboard (agent log + signal board) and a pitch mode with marquee animations and a theatrical "reveal" sequence

The agent autonomously decides which tools to call and in what order — it's not a fixed pipeline. Claude reasons about which shows deserve deeper investigation and stores only the signals it finds credible.

Challenges we ran into

  • Getting the agent loop right. Claude's tool-use cycle needed careful prompt engineering to stay focused on Off-Broadway shows and avoid hallucinating signals. Early runs would confidently "find" awards that didn't exist.
  • Streaming architecture. Piping real-time agent tool calls, ClickHouse writes, and token-by-token brief generation through SSE to the frontend required an async queue pattern and careful state management to avoid race conditions.
  • Signal quality vs. speed. A full deep-dive analysis is thorough but slow; a quick scan is fast but shallow. We ended up building three distinct execution modes (full, quick scan, brief) to balance demo pacing with analytical depth.
  • Theatrical UX under time pressure. We wanted the reveal to feel like a Broadway moment — marquee borders, slam-in animations, a deliberate pause — which meant hand-tuning CSS animations alongside backend work.

Accomplishments that we're proud of

  • The agent genuinely works end-to-end: it searches the live web, extracts structured signals, stores them with evidence, and produces a ranked prediction — all autonomously.
  • Every prediction is backed by source URLs. This isn't a black-box guess; you can click through and verify every signal.
  • The pitch mode UI makes a data product feel like theatre. The 3-second pause after the reveal lands every time.
  • We built a seed dataset of historical breakouts (Hamilton, Hadestown, A Strange Loop) that validates our signal framework — all three scored 5/5 on our signal types before they transferred.

What we learned

  • Agentic AI is most compelling when the agent has real tools that hit real data — mocked demos don't convince anyone. Nimble + ClickHouse gave us a live, verifiable pipeline.
  • Prompt engineering for tool use is a different discipline than prompt engineering for text generation. The system prompt needs to constrain behavior, not just output.
  • Theatre and tech have more in common than we expected. Pacing, timing, and the moment of reveal matter just as much in a demo as they do on stage.

What's next for Hidden Gem

  • Automated monitoring. Schedule the agent to run nightly and alert subscribers when a new show crosses a signal threshold.
  • Expanded signal types. Social media buzz, ticket resale price trends, and casting announcements (a known Broadway star joining an Off-Broadway production is a strong transfer signal).
  • Historical backtesting. Run the agent against archival data from 2013–2025 to measure predictive accuracy and refine signal weights.
  • Regional expansion. London's West End has its own pipeline (Donmar Warehouse, Young Vic, Almeida) — the same framework applies with different venue and award inputs.
  • Public-facing product. A newsletter or app where theatre fans and industry professionals can track which Off-Broadway shows are building momentum — think "the Bloomberg Terminal for theatre."

Built With

Share this project:

Updates