Arc Engine

An agentic GenFX supervisor. An idea becomes a script, a shot list, reference sheets and approved footage - with every decision, cost and refusal on the record.

Comment

Inspiration

Generating a shot is easy. Making it belong in a movie is the problem.

I spent 18 years in VFX — compositor, pipeline TD, ML engineer. The job that actually gates a sequence isn't generation, it's dailies: a supervisor watches the day's footage and says which takes hold up against everything already approved. Nobody has built that for generative video. Every tool races to make one beautiful shot; none of them can tell you whether shot 3 contradicts shot 2.

So Arc Engine is a supervisor, not a generator.

What it does

An idea prompt becomes approved footage, with a human gate at every step that costs money or becomes canon:

idea → script → [approve] → breakdown → [approve] → shots + reference sheets → [lock as canon]
                                                                    ↓
                                    plan → generate → watch → critique → revise → [approve]
  • Story agent turns a one-line idea into a shootable script.
  • Breakdown agent turns the approved script into sequences, shots and the assets they need — writing into the same brief field a human would type, so nothing downstream changes.
  • Asset-sheet agent generates character/prop/environment reference sheets. They arrive unlocked — invisible to every agent until a person locks them as canon.
  • Planner queries approved continuity state from ClickHouse through the official ClickHouse MCP server, as its own tool call, before writing a prompt.
  • Generation adapter calls Veo 3.1, image-conditioned on locked references.
  • Critic reviews labelled frames in order — not just first and last — and returns pass / warning / fail / not_applicable per continuity axis.
  • Approval gate is deterministic, not agentic: approved only when every hard-fail axis clears. Revision rounds are capped; on cap-out a shot escalates to needs_human rather than looping or rubber-stamping.

Everything is on the record: model, tokens, cost, latency, every verdict, every human veto and its reason.

How I built it

Next.js + FastAPI + Postgres + ClickHouse + MinIO/R2, Gemini for planning and multimodal critique, Veo 3.1 for generation. Typed contracts mirrored in Zod and Pydantic; malformed agent output is a hard stop, never a silent pass.

The numbers on the landing page are read live from agent_decision_log. $31.18 across 129 real agent calls. None of it is written by hand.

Challenges I ran into

"Latest" and "approved" are different things. Writing a version's verdict straight onto the shot meant a re-critique of an old version silently moved a shot's status. Fixed with one mandated helper in both languages.

The cost ledger was lying. Every agent logged candidates_token_count as its output. Thinking tokens bill at the output rate and aren't in that field — a real call reported 568 candidate tokens and 2611 thinking tokens, so logged cost was 4.4× under. That mattered more than an ordinary bug because the budget ceiling refuses runs against that number. An undercounting ledger spends past a cap you set and believe is holding.

A scored axis the critic can't see is a phantom defect. Scoring hero_prop on a shot where no prop is visible invents failures. The critic now declines an axis it cannot observe, and the gate counts declined axes separately — so "approved" can never quietly mean "nothing was checked".

The critic is not deterministic. Re-critiquing the same footage returned approved where the first pass returned revise — and the stricter pass was the more accurate one; it caught the lantern in the wrong hand. Both verdicts are on the record. I'd rather ship that honestly than pretend the judge is stable.

Accomplishments I'm proud of

A shot generated from nothing but an idea prompt — script, breakdown, reference sheet, locked as canon, planned against it, generated, critiqued, approved — where no human authored a word of the brief.

And the refusals. The system says no in ways that cost nothing: it refuses a tier that can't use your locked references before the planner is billed; it refuses a batch that would breach the ceiling before the first call; it refuses to rewrite a shot that already has paid-for footage.

What I learned

Guardrails only work if they're mandatory. Three separate bugs were the same shape: a helper existed, and the next caller hand-rolled the original defect. Every fix now ships with a test that enumerates callers and fails on the one that forgot.

What's next

Splitting continuity axes by the kind of question each one is — face as a measurement (embedding distance), outfit and background as judgements (vision model). Identity is the least reliable finding precisely because it's the one axis that isn't a language question.

Built With

Share this project:

Updates

Submission history