Churnalyst

YouTube shows where viewers drop off, not why. Churnalyst uses AI to diagnose exact retention cliffs and generate one-click YouTube Studio trim lists & script rewrites.

Comment

🎬 Churnalyst: The Automated YouTube Retention Explainer

Inspiration

Every creator and video editor knows the exact feeling of opening YouTube Studio, staring at a steep retention drop-off at 02:14, and desperately wondering: "Why did 20% of my audience just leave?"

YouTube Analytics is incredible at showing you where viewers leave, but it never tells you why. Creators spend hours scrubbing through timelines, manually comparing spoken words against retention dips, and guessing whether viewers dropped off because of a boring tangent, poor pacing, or an awkward sponsor transition.

We built Churnalyst to eliminate this post-upload busywork. We wanted to build an automated diagnostic engine that takes raw viewership curves, finds the exact moments of audience churn, correlates them with the spoken transcript using AI, and gives creators instant, actionable fixes in under 15 seconds.

What it does

Churnalyst automates the entire post-upload post-mortem workflow for YouTube creators:

  1. Rolling Z-Score Anomaly Detection: Rather than relying on arbitrary percentage drops, Churnalyst fits an exponential baseline decay curve to the video's retention data and computes the 1st derivative (drop velocity) to isolate statistically significant retention cliffs ($Z\text{-score} < -2.0$).
  2. Context-Aware AI Diagnostics: Cross-references each statistical drop with the exact words spoken at that second using Google Gemini Flash to explain the root cause (e.g., tangent, broken hook promise, tone shift, pacing drag).
  3. First-30s Hook Audit: Compares the video's title and thumbnail promise against the opening 30 seconds of speech to score hook alignment and spot immediate viewer bounce.
  4. Sponsor Drop Tax Isolation: Automatically detects paid integrations, calculates the exact audience loss during the ad read, and scores sponsor retention impact.
  5. One-Click YouTube Studio Trim EDL: Generates a copy-ready Edit Decision List (EDL) with exact timestamps (start - end) so creators can immediately trim dead weight inside the YouTube Studio editor without re-rendering or re-uploading.
  6. Instant Fix Prompts & Wellness Digest: Provides one-click copyable LLM prompts to rewrite weak script segments and pins 3–5 strict creator rules for the next video upload.

How we built it

  • Data Pipeline: Integrated with the YouTube Data API v3 and YouTube Analytics API v2 for first-party channel viewership data, secured by a Google OAuth 2.0 Web Flow.
  • Resilient Caption Extraction: Engineered a zero-download direct subtitle parser using Android/iOS player client bypasses to extract timestamps in under 1 second, with a multimodal Gemini Flash Audio fallback for videos without captions.
  • Statistical Modeling: Built custom NumPy/Pandas algorithms to calculate rolling exponential decay curves, velocity gradients, and Z-score thresholding.
  • AI Diagnostics: Powered by Google Gemini Flash for high-speed semantic alignment, hook evaluations, and script rewrite generation.
  • Frontend & Visualization: Built with Streamlit and interactive dark-mode Plotly charts featuring unified hover tooltips that display time, audience lost, severity score, and spoken transcript context.

Challenges we ran into

  1. YouTube Anti-Bot & SABR Streaming Protections: YouTube's recent enforcement of bot challenges on automated media downloads caused traditional scraping and yt-dlp audio extractors to hit HTTP 429 / bot verification roadblocks. We overcame this by building a multi-layer extractor that leverages mobile player client bypasses and parses raw json3 caption feeds directly, reducing extraction time from minutes to milliseconds without triggering bot detection.
  2. PKCE State Management in Reactive Streamlit: Implementing Google OAuth 2.0 with PKCE in a reactive framework meant that page reloads created fresh Flow instances missing the initial code_verifier. We resolved this by persisting OAuth state and verifiers across session state re-runs.
  3. Separating Natural Decay from True Churn: Retention curves naturally slope downward as videos progress. Distinguishing between expected viewer drop-off and actual creator mistakes required tuning our statistical rolling Z-score algorithm against first derivative velocity curves.

Accomplishments that we're proud of

  • Producing Actionable Deliverables (Not Just Charts): We didn't want another passive dashboard. We're proud that Churnalyst outputs concrete deliverables: downloadable YouTube Studio Trim EDLs, exportable Markdown Repair Plans, and one-click Script Rewrite Prompts.
  • Sub-Second Transcript Alignment: Achieving near-instant transcript retrieval and alignment across any public or private YouTube video without requiring heavy audio rendering pipelines.
  • Zero-Friction Test Mode: Creating a resilient pipeline where judges and creators without OAuth access can instantly evaluate any public video by synthesizing retention curves against real metadata and transcripts.

What we learned

  • How to combine rigorous statistical anomaly detection with multimodal LLM reasoning to prevent hallucinations and ground AI insights in mathematical reality.
  • The depth and nuances of Google's YouTube Data and Analytics APIs, token exchange lifecycles, and modern video streaming protocol challenges.
  • How much creator burnout is caused by post-upload anxiety—and how automating retention diagnostics can give creators clarity and confidence for their next upload.

What's next for Churnalyst

  • Multimodal Video Frame Analysis: Sampling video frames at exact drop timestamps to detect visual mistakes (bad graphics, jump cuts, unreadable text, boring slides) alongside the audio transcript.
  • Direct YouTube Studio API Trimming: Integrating with YouTube's editing APIs to automatically apply proposed trim edits directly from the dashboard.
  • Automated A/B Test Hook Generator: Automatically generating 3 alternative script hooks and thumbnail concepts tailored to eliminate the specific drop-off patterns of past videos.

Built With

Share this project:

Updates