Tonight’s Pick

Inspiration

"What should we watch tonight?" is a question LLMs answer confidently and badly: they suggest titles that don't exist, that you've already seen, that aren't on your services, or that nobody actually enjoyed. We wanted an agent that answers it with evidence: what ordinary viewers really thought, checked by the server, and fast and cheap enough to run every night.

What it does

Tonight's Pick recommends three movies or shows you haven't seen, that fit your taste, and that real viewers liked.

• The Real People Score:
  • We read public comments (YouTube, Bluesky, TMDB reviews), and NVIDIA Nemotron classifies each one: liked, mixed, disliked or not an opinion.
  • The result is a share of viewers who liked the title ("94% liked, ±3 points, 314 opinions"), compared within its genre, plus a summary and real linked quotes.
• An agent that can't cheat. Ask "a fun family movie about Halloween, under 2 hours" and the Nemotron 3 Super agent calls tools for taste, titles, constraints and scores. The server then checks every answer:
  • every pick must come from a tool result;
  • topic, runtime, rating, services and language are enforced;
  • nothing you've seen or disliked;
  • a reception floor;
  • the "why it fits you" line is built from verified facts.
• "How we picked this" shows every Nemotron tool call, every data call, and the tokens and cost of that exact answer.
• Also: Pick together for 2–5 people, taste match, voice via MCP, and a per-profile language and streaming-service filter.

How we built it

Every model call runs on NVIDIA Nemotron through Nebius Token Factory, via the OpenAI-compatible API, behind one module (apps/api/src/llm) that meters every call into a usage ledger with daily caps.

Where the calls happen in the code:

• apps/api/src/pipeline/classify.ts: the classifier and second opinion.
• apps/api/src/agent/loop.ts: the agent.
• apps/api/src/pipeline/summary.ts: the summaries.

Measured costs and speed:

• Agent answer: median about 1 cent (≈1.1¢ over our last 43 answers) and 10–30 seconds.
• Scoring a new title: about 5¢, mostly the second opinion.
• Turning reasoning off for agent steps cut answer time 2–3× (e.g. 48s → 33s, 91s → 28s) with the same picks. The server enforces constraints and writes the reasons, so the model no longer needs to deliberate over them.

The rest of the stack:

• Data and APIs: Fastify, Prisma and Postgres (Neon), pg-boss for nightly pre-scoring.
• Sources: TMDB (titles, keywords, where to watch), Qloo (taste), YouTube Data API and Bluesky.
• Web: Next.js on Vercel.

Challenges we ran into

• Reasoning tokens ate the output. With reasoning on and a 400-token limit, about half our summaries came back empty: the reasoning used the whole budget. We turned reasoning off for summaries and agent steps, and measured that quality held.
• Telling opinions apart from complaints about opinions. "Fans who hate this are wrong" isn't a dislike. We wrote a cls-v4 prompt with 16 worked examples, added the Super second opinion on hard cases, and tagged every label with the model and prompt version, so changes trigger a re-score.
• Models invent reasons. The agent once called a dark sci-fi thriller "light-hearted and festive". Now the server writes every "why it fits you" from data it checked: shared taste tags, matched genres, or a genuinely similar title you liked.
• Reliability under caps:
  • concurrency limits with 429 backoff;
  • a visible "scoring paused until…" state instead of silently falling back to a weaker model;
  • recovery after restarts.

Accomplishments that we're proud of

• The agent's answers are verified, not trusted. The server enforces every constraint and topic, and rejects picks that didn't come from tool results.
• Full provenance per answer: model, steps, data calls, tokens and cost, visible to users.
• Cheap enough to run a real product: about 1¢ per answer, and nightly pre-scoring under a fixed budget.
• 120+ automated tests across the scoring math, the classifier's handling, server checks, voice rules and quotas.

What we learned

• Split the work by model size: Nano for volume, Super for judgment. Most of the classifier's errors were in a small set of cases worth a second look.
• Measure before keeping a model setting. Reasoning looked like free quality; in our pipeline it mostly cost time, tokens and, in one case, the entire output.
• Put the rules in code, not the prompt. The more the server enforced, the less the model had to "think" and the faster and more accurate it got.

What's next for Tonight's Pick

• Publish the evaluation:
  • classifier agreement with hand labels;
  • the agent vs. a plain chatbot on the same Nemotron model (hallucinated titles, seen titles, constraint violations, reception of picks, latency, cost).

The harness is in docs/eval.

• A Nano-first agent for simple requests, keeping Super for complex ones.
• More sources and languages, and a subscription (3 free picks, then $1.99/month).

The same codebase is also entered in the Amazon (Alexa+) and Qloo hackathons. MIT licensed: github.com/gunnerhowe/TodaysWatch · Live: https://todays-watch.vercel.app

Built With

• bluesky
• docker
• fastify
• model-context-protocol
• nebius-token-factory
• nemotron-3-nano
• nemotron-3-super
• neon
• next.js
• node.js
• nvidia-nemotron
• openai-sdk
• pg-boss
• postgresql
• prisma
• qloo
• railway
• react
• tailwindcss
• tmdb
• typescript
• vercel
• vitest
• youtube-data-api
• zod