Render, Look, Fix: Nemotron Makes Motion Graphics

Nemotron 3 ranked last on Code2Video, where models get one shot and no tools. I gave Nemotron 3.5 Super VL a render, look, fix loop with blind review, and it made the motion graphics itself.

Comment

Inspiration

Artificial Analysis recently published Code2Video, a benchmark where models turn a design brief into a HyperFrames motion-graphics video (HTML + GSAP) in a single call, with no tools and no chance to see what they rendered. Nemotron 3 Ultra finished 27th of 27, at 304 Elo against 1536 for the leader.

Motion designers don't work in one shot. They render, look at the result, and fix it. I wanted to know whether a newer, low-cost open model, Nemotron 3.5 Super VL on Nebius Token Factory, could do good work if it got the same loop. If it can, that is a cheap option for motion graphics that nobody has measured yet.

What it does

The project runs a Code2Video-style brief through Nemotron 3.5 Super VL twice:

  1. One shot: one call, no tools, the benchmark's condition.
  2. The loop: Nemotron writes the composition, renders it, reads frames from the encoded video, gets a blind review, fixes the defects it found, and repeats.

The worked example is an 8-second app promo ("Tidepool") with generated tide-pool footage, a phone mockup, a drawn tide curve and a headline. The submission video was built the same way: Nemotron wrote the storyboard, script, scene compositions and reviews from my direction notes.

How I built it

I started from media tools I had already written: Media Tooling (ElevenLabs narration with word timings, generated-media requests and take selection) and TTV Pipeline (text/image-to-video behind an approve-then-execute handoff). HyperFrames renders the video and Pi runs the agent loop.

Most of the work was new code to make those tools usable by a different model:

  • A Pi provider extension for the Nebius endpoint, with a usage ledger that stores metadata only.
  • An adapter for Nebius's limit of eight images per request. Visual review replays many frames, so it sends the newest eight and marks older ones as omitted.
  • cues.py: animations are timed to markers in the narration script, not to hand-typed seconds.
  • blind-review.py: a fresh Nemotron call that sees only the brief and frames from the encoded video.
  • check-delivery.py: gates for cue sync, text overflow, long static holds, generated footage in the final encode, and a review on all five Code2Video axes.
  • Always-loaded rules, craft guidance and a review rubric matching the Code2Video judge's axes (Engagement, Prompt-Intent, Composition, Temporal, Craft).

Challenges

  • The first two films failed my own review. Words flashed on screen seconds away from the narration, text ran off the frame, and the generated footage was dropped in full-frame under a caption. Each of these became a rule and an automated check.
  • One shot at high reasoning effort produced nothing. Nemotron spent all 32,768 output tokens reasoning and returned no HTML. At medium effort it returned a complete composition (27,673 tokens, about $0.03).
  • The model's review of its own work was wrong. After reading all eight frames of the first loop render, it scored Prompt-Intent 5/5. The render had no tide curve, no "Low tide 4:12 PM" label and no phone, and the wordmark was printed over the headline. A fresh call with only the brief and the frames found all of it in 14 seconds for under a cent. The model's vision works; what misled it was having its own plan in context.
  • Provider outages. Nebius returned 502/503 errors for a brief period, so I added retries.

What I learned

Over seven rounds, Nemotron's blind review of the Tidepool example went from Prompt-Intent 2/5 with a blocker and three major findings to no findings at all. I think that final 5/5 on every axis is generous: the phone is small and square and the frame has a lot of empty space. The reviewer catches missing elements well and is soft on composition. Running the real Code2Video judge on these renders is the honest next test.

Inference for the whole project, across three film attempts and 647 model turns, came to about $25 estimated at Nebius list prices. Most of the tokens were re-read context, which I counted at the full input rate.

What's next

  • Submit the loop's renders to the Code2Video judge for a score that doesn't come from Nemotron itself.
  • Run all 114 briefs, one shot vs. loop.
  • Make the toolchain more easily installable.

Built With

Share this project:

Updates

Submission history