Foreshadow

An AI showrunner: a near-miss report → the whole short-film pipeline (script→storyboard→QC→render), budget-governed (~$3) and provenance-signed.

Comment

Inspiration

A warehouse safety manager files the third forklift near-miss this quarter. Corporate's answer is a 2009 training video nobody watches. The next one won't be a near-miss.

A near-miss is the accident's foreshadowing — and foreshadowing is a narrative device. We wanted to turn one into the other: treat a company's own incident log as a screenplay archive, and let an agent crew make the film people will actually watch — your aisle, your forklift — for the price of lunch instead of thousands of dollars and six weeks.

What it does

Foreshadow is an AI Showrunner. Drop in a near-miss report and a crew of Qwen agents makes a 90-second cinematic safety film:

  • a Screenwriter (qwen3.7-max with thinking) turns the OSHA-style narrative into a three-beat drama;
  • a Line Producer prices every storyboard shot and allocates render tiers under a hard dollar budget — a knapsack allocator over narrative-weight × tier-cost, with a live ledger and a regret log for every demotion ("saved $0.90, narrative weight 2/10") and a hard-cap kill switch at 2.5× budget;
  • the Art Department (qwen-image-2.0-pro) fixes the cast with a character sheet;
  • the DP shoots hero moments on wan2.7-i2v and connective tissue on wan2.6-i2v-flash;
  • a QC Critic (qwen3-vl-plus) rejects any clip that fails the shot's safety intent and forces a re-render or demotion instead of shipping it;
  • cosyvoice-v3-plus narrates; ffmpeg cuts.

The budget IS the product. The track asks for output quality under a limited budget, so cost is a first-class, visible mechanic. A budget sweep shows quality scaling from ~50% (Ken-Burns stills) at $2 to ~92% (hero tier) at $8. A demo film's ledger closes at $2.71 of a $4.00 budget, versus $5–15k and 6–8 weeks for a conventional safety video.

Because safety films become evidence, every artifact is provenance-signed: SHA-256 leaves (each carrying its render task_id) roll into a Merkle root signed with Ed25519. foreshadow verify film.mp4 manifest.json — or the live /verify endpoint on Alibaba Function Compute — re-derives the chain. Incident uploads are ECIES-sealed at rest. Four invariants are tested, including "no QC-rejected clip appears in the final cut" and "1-byte tamper fails verification."

How we built it

Foreshadow is a Python 3.12 CLI (render · replay · verify · bench · plan · preview) over an 11-stage pipeline with SQLite and a committed fixtures/cache/ for byte-identical replay, seeded with three deterministic OSHA-300-style incidents.

It is built on Qwen Cloud — eight distinct Qwen surfaces on one bill: qwen3.7-max (screenplay + structured JSON output for ShotPlan / BudgetDecision / QCVerdict), qwen-image-2.0-pro (art department), wan2.7-i2v and wan2.6-i2v-flash (async render tiers), qwen3-vl-plus (frame-grounded QC), cosyvoice-v3-plus (narration), and the Batch API (−50% on storyboard fan-out). Take Qwen Cloud out and this becomes four vendors, an async render queue, and a cross-vendor cost normalizer — the single-bill ledger that is the demo becomes impossible.

The transport is swappable: FakeQwen (default, deterministic, zero-key — the graded path) vs LiveQwen behind DASHSCOPE_API_KEY. Chat surfaces are fully implemented on the OpenAI-compatible endpoint and a real DashScope call is smoke-verified; image/video/TTS surfaces are payload-complete builders that raise LiveSurfaceNotVerified until a key is present.

It is deployed live on Alibaba Cloud Function Compute (managed python3.10 runtime) at https://foreshadow-txebjackop.ap-southeast-1.fcapp.run, exposing /health (liveness), /verify (cloud-side signed-ledger replay + Ed25519/Merkle re-verify + invariants I1–I4, byte_identical_cache: true), and /run?incident=forklift (full pipeline replay → ledger $2.71, budget mix, QC counts, Merkle root). The deployed endpoints serve the same offline-deterministic FakeQwen path graded by the tests, so the cloud response is identical to a local foreshadow replay. Quality is enforced by 421 tests at 100% coverage (session-wide socket guard), ruff, a socket-guarded offline judge proof, and a 5-stage GitHub Actions pipeline with CodeQL, Dependabot, pip-audit, and TruffleHog.

Challenges we ran into

  • Turning a budget into a mechanic, not a limit. We built the Line Producer as a knapsack allocator over narrative weight, with a regret log and a 2.5× kill switch, so the cost trade-off is visible and testable rather than a hidden cap.
  • Making provenance honest about what it does and doesn't prove. Signing proves pipeline integrity, not narrative truth — a fabricated report yields a validly-signed fabricated film. We state that limit up front and test the integrity claims (1-byte tamper of every committed artifact flips verification to FAIL).
  • QC that demotes instead of pretends. wan clips occasionally drift props between shots; the qwen3-vl-plus critic rejects the clip and forces a re-render or demotion rather than shipping it.
  • Deterministic media for replay. The graded offline path writes a signed edit-list stub as film.mp4 for byte-identical replay; foreshadow preview renders a real playable animatic (Pillow + imageio-ffmpeg), labeled honestly as an animatic — real AI video only runs behind a key.

Accomplishments that we're proud of

  • $2.71 vs $5–15k — a personalized safety film from a company's own incident, on a lunch budget.
  • 421 tests at 100% coverage, fully offline via a session-wide socket guard.
  • Cryptographic provenance (Ed25519 + Merkle) with tested tamper-evidence, purpose-built because safety films become evidence.
  • A budget-as-product mechanic with a published $2/$4/$8 quality sweep — the track's constraint made into the demo.
  • Deployed live on Alibaba Function Compute with a cloud-side /verify that re-derives the whole signed chain.

What we learned

When you make cost a first-class, visible variable — a ledger, a regret log, a sweep — a budget stops being a wall and becomes the most interesting part of the product. And provenance is only credible if you're precise about its boundary: signing proves the pipeline didn't tamper, not that the story is true, and saying so is what makes the signature worth anything.

What's next for Foreshadow

  • Archive renders to Alibaba OSS and add the Next.js war-room UI (agent lanes + live ledger + player) on top of the already-live Function Compute backend.
  • Execute a full captured live keyed Qwen run so the provenance manifest carries real DashScope task_ids and the ledger reflects real spend (a single DashScope call is already smoke-verified).
  • Ship an x402 pay-per-film API and publish the CLI to PyPI.

Built With

  • alibaba-cloud
  • batch-api
  • codeql
  • cosyvoice-v3-plus
  • dashscope
  • ecies
  • ed25519
  • function-compute
  • github-actions
  • merkle-tree
  • pydantic
  • pynacl
  • pytest
  • python
  • qwen
  • qwen-image-2.0-pro
  • qwen3-vl-plus
  • qwen3.6-flash
  • qwen3.7-max
  • ruff
  • sqlite
  • wan
  • wan2.6-i2v-flash
  • wan2.7-i2v
Share this project:

Updates