Merchant Marquee

Merchant Marquee turns product photos into polished AI video ads, handling scripts, shots, voiceover, captions, and editing so small sellers can market faster and affordably.

Comment

Inspiration

Small Etsy and Shopify sellers can't compete with big brands on video ads. A single 15-30 second product ad from a freelance studio costs $300-$1,500 and takes days to turn around — so most small sellers just don't run video ads at all, even though video consistently outperforms static photos across every major ad platform.

We didn't want to build "type a prompt, get a video" — that's generic and doesn't solve a real problem. We wanted an agent crew that behaves like an actual small production team: someone who reads the product photos and figures out what's actually true about the product, a writer, a critic, a director, a producer keeping the budget honest, and a continuity checker making sure the footage doesn't drift from the real product. Merchant Marquee is that crew — Qwen-powered agents that write, critique, direct, and shoot a real ad from nothing but 2-3 product photos and a one-line brief.

What it does

A seller uploads 2-3 real product photos and writes one line about their product. From there, Merchant Marquee runs the whole thing end to end:

  • A Truth Agent (Qwen-VL) reads the photos and extracts specific, grounded facts about the product — material, color, texture, construction details — never generic category filler, and never negatives unless the seller explicitly asks for an "authentic" angle.
  • A Concept Agent writes four genuinely distinct script variants, each forced into a different narrative framework, and a Critic Chain (hook, pacing, completion, CTA, tone) scores every variant on a weighted rubric.
  • A Meta-Critic cross-pollinates the strongest pieces of each variant into one merged script, and an independent Merge Coherence Validator — a separate node, not the same agent grading its own work — checks the merge for voice consistency and promise-payoff match before it's allowed to proceed. A Copy Editor repairs voice mismatches in place; a real fallback path exists if repair fails twice.
  • A Treatment Agent turns the winning script into a director's treatment — persona, color story, pacing — every beat justified by a real quoted line from the script and a real cited product fact.
  • A Shot-List Agent builds a camera-literate shot list (lens, movement, composition) where every shot's justification is deterministically validated — no shot is allowed to be justified by a generic product category, only by a real fact and a real script line.
  • A Budget Gate allocates real dollar cost per shot (grounding-weighted, not an equal split) against a hard cap, with an automatic, deterministic reduce-and-redistribute pass if the shot list comes in over budget.
  • A Video-Gen Node fans every shot out in parallel to Wan2.6 (image-to-video, never text-to-video, so every clip is anchored to the seller's real product photo) — and if any single shot's API call fails or times out, it routes straight to a Ken-Burns fallback clip instead of blocking the rest of the ad.
  • A Continuity Agent compares every generated frame back against the source photos and flags drift; a capped auto-retry loop fixes minor drift automatically, and a genuine human-in-the-loop interrupt pauses the run for a real person to approve, retry, or accept the fallback when retries are exhausted.
  • A Voiceover + Caption Agent generates narration and burns in captions in parallel with the shoot, and an Assembly Agent stitches the final cut and exports it in 9:16, 1:1, and 16:9 — one shoot, three formats.

Every one of those decisions — every score, every dollar spent, every drift check — streams live to a dashboard, so nothing happens in a black box.

How we built it

Two of us split every phase of the build down the middle rather than splitting by frontend/backend, so neither of us got siloed for the whole project. The backend is a LangGraph state machine running on FastAPI, with Qwen (text, vision, and TTS — across two DashScope regions, since TTS access turned out to be region-scoped differently than everything else) and Wan2.6-i2v for video generation. The frontend is Next.js, streaming the whole pipeline live over WebSocket. Everything runs on Alibaba Cloud — ECS for compute, OSS for asset storage (signed URLs on a private bucket, not public access), and a managed Postgres instance for job state and checkpointing.

We agreed a handful of shared contracts up front — the state schema, the event schema, the shot-list schema, the human-review interrupt contract — and treated them as real interfaces: draft together, sign off together, only extend additively afterward. That discipline is what let two people build 15+ interdependent agents in parallel without constantly blocking on each other.

Challenges we ran into

The honest ones, not the polished ones:

  • Video-gen product identity drift. Early on, a Meta Quest headset photo produced a generated clip of "a phone on a phone stand" — the model's training priors overriding an under-specified reference image. We root-caused it through real research (not guessing), added a mandatory whole-object "form factor" grounding fact, hardened the negative prompt, and added a categorical same-object identity check on an early frame of every generated clip — with a hard-fail route straight to the fallback if identity checking fails twice in a row.
  • Scripts that led with the product's own flaws. An early version of the pipeline kept producing ad copy that opened on a scratch or a defect, or framed the viewer's own insecurity as the hook. We rebuilt the script generation around real narrative-structure research — open on a human moment, arrive at the product, close on one earned ask — with a deterministic rhyme detector and a flaw-led-hook detector backing up the prompt, not just trusting the model to behave.
  • A silent single-point-of-failure bug in our merge validator could crash an entire job with no graceful recovery if zero script variants survived scoring — we didn't just patch the crash, we traced it back to why variants were failing in the first place and fixed the real upstream cause too.
  • Getting a real job to complete end-to-end for the first time took fixing a chain of five separate bugs across the stack — a photo-upload race condition, a missing dotenv load, a stale recursion limit that was correctly set in our test suite but never ported to production, a state-key mismatch that silently emptied dashboard panels on reopening a past job, and a CORS policy that only trusted localhost. Every one of them was found by actually running the real pipeline against real data, not by code review alone — that habit is probably the single biggest reason this project works today.

Accomplishments that we're proud of

  • A real seller-uploaded photo genuinely produces a real, watchable, correctly-formatted ad video end to end — not a demo path, the actual pipeline.
  • Every agent decision is inspectable: real critic scores with real reasoning text, a real budget ledger, a real continuity drift score per shot — nothing is a black box.
  • A genuine human-in-the-loop interrupt that pauses a real LangGraph run mid-execution and resumes correctly on all three human decisions.
  • Graceful degradation everywhere it matters: a failed video-gen call never blocks the ad, an over-budget shot list gets deterministically trimmed instead of silently overspending, and a failed job now fails visibly instead of crashing the connection.

What we learned

That "it compiles and the tests pass" and "it actually works" are two different claims, and the gap between them is where the real bugs live. Repeatedly, our test suite was green while the real pipeline was silently broken — a TTS response field that was a dict instead of an object, a WebSocket event that had never once fired in production, a recursion limit that existed correctly in one test file and nowhere else. The fix, every time, was running the real thing against real data and checking real evidence, not trusting a passing test or a plausible-sounding summary.

What's next for Merchant Marquee

Chat-based revision — asking for "make the hook punchier" or "shot 2 is too dark" and having it re-render only the affected shots via a checkpoint fork, instead of a full re-run. We also want to push further on dynamic mid-pipeline budget renegotiation, and to broaden the video-gen identity-grounding work we did for ambiguous products (like the VR headset case) into a more general defense against any product shape a seller might upload.

Built With

Share this project:

Updates