Inspiration ## Inspiration
I wanted the text of a video and discovered there is no good way to get it.
YouTube's official Data API will tell you a caption track exists and then refuse to hand it over: captions.download requires the video owner's OAuth token, so unless you own the channel, the endpoint is useless. The popular Python library works until you deploy it, at which point YouTube starts refusing your datacenter IP. And that is the easy platform. TikTok and Instagram publish no caption file at all.
Every workaround I found solved one platform, worked locally, and broke in production.
What it does
TranscriptFetch is one API that returns timestamped text from YouTube, TikTok, Instagram, X, and Facebook, plus any web page as clean Markdown.
The part that makes it actually work: when a video has no caption track, it transcribes the audio instead. Same endpoint, same response shape, so your code never branches on which method produced the text. You get a joined text field for feeding a model and a segments array with start times and durations for subtitles or jump-to-moment links.
It also ships an MCP server, so Claude, Cursor, and other MCP clients can fetch transcripts as a tool mid-conversation, plus Python and JavaScript SDKs and a batch endpoint for up to 50 videos per call.
How I built it
Frontend and API: Next.js 15 with the App Router and TypeScript, Tailwind for styling, Clerk for dashboard auth with SHA-256 hashed API keys.
Data: Neon Postgres with Drizzle ORM, Redis for caching, Stripe for subscription billing.
The scraper: a self-hosted Python and FastAPI service built on yt-dlp, sitting behind nginx. This is the piece that has to survive contact with reality. It runs as its own container in the same Docker stack.
Transcription: Whisper-class speech-to-text through Groq for the videos with no caption track.
The API is single-sourced from one catalog module, so the OpenAPI spec and every docs page render from the same definition and cannot drift.
Challenges I ran into
Most short-form video has no captions. I tested three Instagram Reels while building this. All three returned no_captions from the caption path. All three transcribed fine once the audio fallback ran. That is the normal case on Instagram and TikTok, not the exception, which means speech-to-text is the primary path on those platforms rather than a backup.
Telling "no captions" apart from "no speech." A dance clip over a licensed track has nothing to transcribe, ever. A talking-head video with no caption track has plenty. The first should fail fast and never be retried or billed; the second should go straight to ASR. Getting that distinction wrong means either charging people for silence or giving up on videos that would have worked.
Platforms block servers. Anything that works on a laptop stops working from a datacenter IP. Solving this properly is infrastructure, not code.
Billing only on success. Failed and blocked videos are free. That sounds simple and means the billing path has to be wired through every failure mode in the pipeline rather than bolted on at the end.
Accomplishments I'm proud of
The MCP server is published in the official Model Context Protocol registry with a DNS-verified namespace, so AI clients can discover it rather than needing manual setup.
I published a CSA STAR Level 1 CAIQ v4 self-assessment, 261 questions answered conservatively, because "we take security seriously" is not a claim anyone should believe without evidence.
Uptime sits at 99.92%, measured by an independent monitor rather than self-reported.
What I learned
The hard part of a scraping product is not the scraping. It is everything around it: classifying failures precisely enough to bill fairly, keeping a caption-first path fast while an ASR fallback runs behind it, and building infrastructure that platforms have an active interest in breaking.
I also learned how much of the "just use this library" advice online quietly assumes you are running on your own laptop.
What's next
Better coverage of the failure taxonomy so callers can act on why something failed rather than just that it did, more SDK surface area, and continuing to make the AI fallback path cheaper so the price of a transcript stays flat whether or not captions existed.
Built With
- fastapi
- nextjs
- python
- react
- typescript
