ChannelIQ

Inspiration

"Analytics dashboard" is the most overdone idea in this space. Charts of views-over-time aren't a product, they're a Tuesday.

The realisation that started this: YouTube Studio can tell a creator which of their videos did well. It structurally cannot tell them about the video they never made. A creator with 20K subscribers has plenty of data and no time to mine it, and the question they actually have isn't "how did I do?" — it's "what should I make next, and why?"

So ChannelIQ answers that question instead. Not a visualisation. A prescription.

What it does

Paste in a YouTube channel. In under a minute you get a report a consultant would charge for:

• A diagnosis in two or three plain sentences, citing your actual numbers
• Three ranked video concepts, each with the evidence behind it
• A title formula, demonstrated by rewriting one of your own recent titles
• A runtime band and a publish slot, with an honest label for how much of it the data actually supports
• Competitor whitespace — topics out-performing on adjacent channels that you've never covered
• A "stop doing" list and a checklist for the next upload

There are charts, but they sit at the bottom as proof. The prescription is the product.

It runs end to end with zero API keys using bundled demo channels. Those channels are synthetic and labelled as such in the report — publishing fabricated view counts under a real creator's name would be misleading, so the demo channels are fictional but carry genuine, recoverable statistical structure. Live mode analyses any real channel you type in.

How we built it

Four agents, and only the last one touches an LLM:

Data Collector    -> YouTube Data API v3        (no LLM)
Pattern Analysis  -> deterministic correlation  (no LLM)
Whitespace        -> competitor gap detection   (no LLM)
Thumbnails        -> Gemini vision, perception only
Strategy Writer   -> the only generative call

The central design decision: the LLM narrates numbers, it never computes them. By the time the Strategy Writer runs, every fact already exists as a computed figure with a confidence grade. That boundary is enforced, not requested:

1. Server-enforced structured output via Gemini's responseSchema, so a malformed report isn't a possible failure mode
2. A numeric claim guard that checks every figure in the generated prose against the numbers the model was actually given — an unverifiable figure triggers a corrective retry naming the specific violations, then a hard fallback
3. A deterministic writer producing the same report from the same signals with no LLM at all

Two statistical choices do the real work. Raw view counts aren't comparable — a 400-day-old video has had 100x longer to accumulate views, while the channel was smaller back then. So every video is scored against the median of its nearest neighbours by publish date, which cancels both biases at once. And because runtime is the strongest confounder (the deep-dives are long, the quick tips are short), title/topic/timing comparisons divide out the video-length effect first — otherwise "long videos win" masquerades as "this topic wins".

Challenges we ran into

Being statistically correct and completely wrong. The worst output this project ever produced was a recommendation to make videos about #ashusir #scienceandfun — a competitor's own branding hashtags — graded high confidence. The maths was right; the advice was gibberish. Four causes: the tokenizer kept # as a word character, auto-suggested competitors weren't required to actually be out-performing us (that one had 0.91x our median), the self-name filter only worked word-by-word, and duplicate labels weren't deduplicated. Earlier versions had also recommended "make videos about Shorts" to a tech channel and surfaced a prank channel as a competitor to Linus Tech Tips.

The guard didn't survive a provider switch. Migrating from one LLM provider to another broke fabrication detection in three ways: the allow-list collector and the verifier used two different number regexes, neither handled comma-grouped thousands (137,000 parsed as 137), and a 2% tolerance band meant a fabricated number only had to land near any real one — a figure derived by arithmetic from two real values sailed straight through.

The pipeline was unbounded. A fresh live run took 119.5 seconds, 101s of it in narration, because the quota model-walk, 503 backoffs and correction retries compose multiplicatively. On a 60-second serverless function that request just dies.

Serverless caching is a trap. "Warm the cache before demoing" is wrong on Vercel — instances are ephemeral, /tmp is per-instance, and a cold start begins empty. Combined with Gemini's free tier of 20 requests per day per model, a judge on a cold deployment could get templated prose.

A boolean that couldn't tell the truth. The timing recommendation's isHeuristic flag had two states but the derivation had three. The middle case — the weekday is a real finding, the hour is not — got forced to false, so the flag contradicted the prose beside it.

Accomplishments that we're proud of

• The numeric claim guard. Programmatically verifying an LLM's prose against the numbers it was given, with a self-correcting retry and a deterministic fallback. Its test suite pins parsing, acceptance, rejection, and — most importantly — that a fabricated figure spliced into real model output is caught. A guard that never fires is indistinguishable from one that works.
• 62 automated assertions covering the statistics, the wording rules and the safety mechanisms, plus a suite that runs against real model output.
• It degrades honestly at every level. No keys, no quota, stale data, overloaded model, bad JSON, thin evidence — every path produces a real report or a specific, actionable message. Never a raw error or a spinner.
• Quota discipline: 3 API units per 50-video analysis where the naive implementation costs 150.
• It says "I don't know." Timing carries a three-state evidence grade. Thumbnail advice is downgraded from an instruction to "worth testing" when the cohort is thin. The whitespace scan reports zero gaps rather than inventing them.

What we learned

Statistically correct plus garbage advice is the worst possible failure mode — worse than an obvious error, because it's persuasive. Every guardrail here exists because a real channel produced confident nonsense.

A test that cannot fail is worse than no test. One assertion read !ours.isHeuristic || report.isHeuristic, which is vacuously true whenever our own value is false — so it skipped checking exactly the cases where the flag was wrong, while looking like coverage.

Verify the measurement before trusting the conclusion. Twice I diagnosed a serious problem from bad data: a 15-minute "hang" that turned out to be a permissions prompt, and a "41 second" production response that was the HTTP client's stream-reading overhead. Measured properly, production runs in 14-31 seconds.

Determinism is a feature. The precomputed narration bundle is keyed on a hash of the computed briefing, so a Math.round on a week-quantised timestamp — which flipped by a whole week on a few minutes of elapsed time — silently made demo channels take 34s instead of 0.2s.

What's next for ChannelIQ

• OAuth for impressions, CTR and watch time. Public metrics are the real ceiling here. CTR would sharpen the title and thumbnail analysis enormously, and it's the one addition that changes what the tool can conclude rather than just how fast it concludes it.
• Semantic topic matching to replace bag-of-words. Current whitespace detection finds vocabulary gaps, which on a broad channel isn't the same as subjects never covered.
• Track a channel over time and measure whether the recommendations actually worked, closing the loop from advice to outcome.
• Model Shorts and long-form separately. They're bucketed today but have genuinely different distribution mechanics.

Built With

• gemini-api
• google-gemini
• ndjson
• next.js
• node.js
• react
• recharts
• rest-api
• tailwindcss
• tsx
• typescript
• vercel
• youtube
• youtube-data-api