Historical AI Vodcast

History, redirected. An AI Showrunner agent researches, writes, voices, and publishes historical debates as vodcasts. No human in the loop.

Comment

Inspiration

I wanted to create a product where we can contrast the results and to learn from the process. And what better than history? Textbooks tell you that Tesla and Edison disagreed about AC versus DC power, now you can see the argument. The goal from day one was an agent pipeline general enough to produce an episode about any two (or more) figures, on any topic, and just as easily extend beyond history into any kind of scripted dialogue content in the future.

What it does

Give the Historical Vodcast Showrunner a topic and a list of figures. Say, "the AC/DC current war," Nikola Tesla and Thomas Edison, and it autonomously:

  1. Researches each figure, producing confidence-tagged fact sheets grounded in verifiable claims rather than invented history
  2. Writes a turn-by-turn dialogue script, staying strictly within what the research actually supports
  3. Casts voices for each character. Synthesizing a new persistent voice profile the first time a character appears, and reusing that exact voice on every future episode
  4. Generates video for each line of dialogue, using each character's reference portrait to keep their likeness consistent across clips and across episodes
  5. Localizes the finished episode with subtitles in five languages
  6. Publishes the final cut directly to YouTube, fully assembled with captions attached

The entire process is handled by a single HTTP call to a Function Compute endpoint on Alibaba Cloud. No manual editing step in between.

How I built it

The project is an orchestrator (Qwen3.7 Max, using tool-calling) coordinating five specialized agents, each backed by a different model:

Research→Script→Voice→Video→Localization→Publish

  • Research & Scriptwriting: Qwen3.7 Max, run at low temperature for research (accuracy over creativity) and higher temperature for scriptwriting (natural, dramatic dialogue)
  • Voice casting: CosyVoice, over a real-time duplex WebSocket connection, with a voice-enrollment step that registers a persistent voiceId the first time a character speaks
  • Video generation: A fallback chain: HappyHorse first, falling back to Wan 2.7, then Wan 2.6, using reference-to-video mode so each clip stays visually consistent with a stored character portrait
  • Localization: Qwen3-Omni-Flash, generating SRT subtitles in Spanish, Portuguese, French, German, and Japanese while preserving tone and historical register
  • Finalization: Clips are pulled from Object Storage, concatenated with FFmpeg into a single cut, and uploaded to YouTube via OAuth2 with caption tracks attached per language

Character and episode state persist in Alibaba Cloud Table Store. This is what makes voice and likeness reuse real rather than cosmetic. The second time Tesla appears in any future episode, the pipeline looks up his existing voiceId and referenceImageUrl instead of regenerating them from scratch. The whole stack is defined in Terraform: an OSS bucket for generated media, two Table Store tables, a scoped RAM policy for the Function Compute execution role, and an API Gateway trigger in front of it. One terraform apply stands up the entire backend.

Challenges I ran into

The AWS-to-Alibaba Cloud migration was smoother than I expected, but not frictionless. Conceptually, the services map cleanly: Function Compute mirrors Lambda, Table Store mirrors DynamoDB, OSS mirrors S3. But the request shapes, SDKs, and IAM-equivalent permission model are different enough that I couldn't just port code directly. Terraform ended up doing a lot of the translation work for me: writing the infrastructure once as code made the provider switch mostly a matter of swapping resource types, not relearning deployment from zero.

Video generation requires real resilience, not just a single API call. Because I built the entire project using only free-trial tiers without extra credits, I frequently ran directly into hard quota ceilings. Early on, a single quota error partway through an episode meant losing all the work generated up to that point. To survive this constraint, I built an explicit fallback chain: HappyHorse tries first, and on a detected quota or 429 error, the pipeline gracefully falls through to Wan 2.7, and then Wan 2.6. Since these models don't share an identical parameter format for reference-to-video generation, the pipeline handles each model's distinct request body shape separately.

Voice consistency across episodes was harder than voice generation itself. Generating one good voice line is easy; making sure the same Tesla sounds like Tesla in episode five as he did in episode one required treating voice as a persistent state, not a per-request parameter. That meant designing the character store before I designed anything else. Persistence had to come first, not as an afterthought.

Accomplishments that I'm proud of

A pipeline that actually finishes: It's easy to build a demo that generates one impressive clip. It's much harder to build one that goes from a single topic prompt all the way to a published, captioned YouTube video with zero manual steps in between. Watching finalizeEpisode() pull clips from storage, stitch them with FFmpeg, and push the result live was the moment I felt like I'd built a product, not a proof of concept.

Voice and likeness that genuinely persist. The second time a character appears in any future episode, the pipeline doesn't regenerate them; it looks them up. That sounds small in a sentence, but it was the design decision I had to get right before anything else worked, and getting it right means every future episode is cheaper and more consistent than the last.

A video generation step that survives failure. Rather than one hardcoded model call that breaks the whole episode on a quota error, the fallback chain across HappyHorse, Wan 2.7, and Wan 2.6 means a single provider hiccup doesn't cost hours of upstream work. That resilience isn't visible in the output, but it's the difference between a demo that works once and a system I'd trust to run unattended.

What I learned

The video generation quality genuinely surprised me. I went in treating it as the riskiest, most likely-to-disappoint part of the pipeline, and it turned out to be the most impressive. Watching a fully autonomous run go from a one-line topic prompt to a published, subtitled YouTube episode. With no manual editing in between, it was the moment this stopped feeling like a demo and started feeling like an actual product direction.

I also came away with a much better sense of where the real engineering weight sits in an "agentic content pipeline." It's not the individual model calls, those are mostly straightforward. It's the orchestration, the failure handling, and the state that has to survive between one episode and the next. That's the part that makes this a Showrunner and not just a script that calls five APIs in sequence. There's real room to keep improving output quality: tighter prompt engineering per model, more control over cross-clip consistency, better resilience for longer episodes. But as a first full pass at an agentic video production pipeline, I think this is a genuinely powerful foundation to keep building on, for historical figures, and eventually for any story at all.

What's next for Historical AI Vodcast

The biggest limitation right now is that every character exists as a single, fixed version of themselves. Tesla always looks and sounds like the same Tesla, regardless of what year the episode is set in. The next major direction is age-aware character consistency: letting the same historical figure appear at different points in their life: Tesla at 10, Tesla at 21, Tesla in his final years, with a visual likeness and a voice register that actually reflect that age, not just a name tag.

This opens up something I'm genuinely excited about: episodes that include memories or flashbacks within a single continuous scene. An older Tesla recalling a childhood moment, voiced and rendered as the child he was, cutting back to his older self mid-conversation, the kind of layered storytelling a real showrunner would reach for, not just a straight A-to-B dialogue.

Concretely, that means:

  • Extending the character model beyond a single voiceId and referenceImageUrl into an age-indexed set of voice profiles and reference portraits per character, so getOrCreateVoice("Tesla", age=10) and getOrCreateVoice("Tesla", age=65) resolve to genuinely different, age-appropriate assets rather than the same adult voice pitched differently
  • Teaching the Scriptwriter agent to reason about age and timeframe per scene, not just per character; so it can correctly place a flashback in Tesla's childhood while the main scene stays in his later years, and hand off the right age context to voice casting and video generation for each
  • Extending CosyVoice's cloning step to capture age-appropriate vocal registers; a believable child's voice is a different synthesis problem than aging an adult voice down, and I want to get that distinction right rather than papering over it with pitch-shifting
  • Widening the project past historical figures entirely, since none of this age-aware character system is actually specific to history, it's a general foundation for any recurring character across any story, historical or invented

The core pipeline proved the concept works end-to-end. The next phase is making the characters inside it feel like they have a whole life, not just one frozen version of themselves.

Built With

Share this project:

Updates