Sizzle Video AI

Inspiration

Every hackathon demands a ~3-minute demo video. It is the highest-leverage artifact in the submission — and the one builders always make last, at 3am, with no editing skill and no time. Good projects lose to worse projects with better videos. The judges of this hackathon have lived that pain.

The insight: a demo video is not a recording. It is a narrative with a fixed 180-second budget — pain, stakes, mechanism, proof, ask. That is a beat sheet. Beat sheets can be written, storyboarded, shot, and cut. And because the subject is software with a live app, the agent can watch its own output and verify whether it worked.

Sizzle exists so the video ships itself.

What it does

Give Sizzle a GitHub repo URL and (optionally) a live app URL. It:

1. Ingests the README, manifests, and commit history to find the build's narrative arc
2. Writes a ≤180s beat sheet with typed shots and VO lines (qwen3.7-max)
3. Allocates each shot to GENERATE / CAPTURE / RENDER under a token budget
4. Shoots — HappyHorse/Wan for dramatized beats, Playwright + Qwen for product footage, ffmpeg templates for stills
5. Narrates with CosyVoice, assembles with captions
6. Critiques the cut with a multimodal scorer (qwen3.7-plus) that emits typed directives (RETIME, RESHOOT, CUT, REORDER, REWRITE_VO) and recuts until the score plateaus

Every run emits manifest.json + a cost report (cost_per_finished_second). The submission thesis: the video is the product is the proof.

How we built it

Ugly-but-complete first, quality second — because a critic loop means quality compounds without supervision.

• Spine: a strict beat-sheet schema (Pydantic). Everything downstream fills typed slots.
• Budget allocator: default to the cheapest lane that can carry the beat; greedy knapsack over GENERATE candidates; demote losers to stylized RENDER.
• Three lanes: GENERATE (HappyHorse 1.1 / Wan 2.7), CAPTURE (Playwright + VLM scout → product stills → motion), RENDER (deterministic ffmpeg templates).
• Critic loop: sample frames + VO transcript → score against the judging rubric → typed revision directives. Guards: max 3 iterations, monotonic best-cut retention, duration invariant, single-reshoot then demote.
• Infra: Cloudflare Worker at sizzlevideoai.com as the public edge; Alibaba Cloud Function Compute (Singapore) runs the pipeline; OSS stores shots, cuts, and manifests; all inference via Qwen Cloud / DashScope intl. MCP tool: make_demo_video(repo_url, app_url).

Proof files: infra/alicloud/pipeline_worker.py, infra/alicloud/render_worker.py.

Challenges we faced

• Alibaba Cloud account / KYC — classic hackathon blocker. Solved hour zero before writing product code.
• Capture on arbitrary repos — full GUI drive + screen-record is brittle. We scoped to web apps and built a scout → screenshot → HappyHorse i2v path with VLM acceptance checks and RENDER fallback.
• HappyHorse quality & clip length (3–15s) — continuity via last-frame i2v seeding (r2v); cold-open demotes to motion graphics when generation disappoints.
• Critic oscillation — hard caps, keep the highest-scoring cut ever (not the last), one RESHOOT then demote.
• Audio double-source — HappyHorse native audio vs CosyVoice VO. Settled per-shot in the schema (vo: null means native audio wins).
• The meta-risk — if Sizzle's own demo video is bad, the presentation score dies. Human-cut fallback planned; the product thesis still commits to the closed loop.

What we learned

A demo video agent is only as good as its contracts: typed shots, typed critic directives, and a solver for token budget — not vibes. Multimodal models become agents when you close the loop with a verifiable predicate (did this frame satisfy the acceptance condition?). Routing expensive generation only where narrative demands it is the difference between a toy and a product. And shipping on Alibaba Cloud + Qwen Cloud under a weekend deadline teaches more about production ML systems than any tutorial.

Built With

• alibaba-cloud
• cosyvoice
• dashscope
• happyhorse
• python
• qwen
• qwen3.7-max
• qwen3.7-plus
• typescript
• wan