Inspiration

Language models can write robot simulation code that runs cleanly but never completes the task, and then report success anyway. We wanted a pipeline where success is measured by execution, never claimed by the model.

What it does

SimSmith turns a plain-language robot task into a verified MuJoCo script. Nemotron (served on Nebius Token Factory) writes five candidate scripts. A GPU-free pre-flight check rejects bad code, each survivor runs headless on a Franka Panda arm, and a deterministic verifier measures the final state itself. The script never grades its own work. Failures get up to two repair rounds, and verified scripts go into a library. Out-of-scope requests get the closest supported template plus an explicit statement of the gap: never a dead end, never a fake success.

How we built it

Nemotron 3 Super on Nebius Token Factory for planning, generation and repair
MuJoCo with the Franka Panda (mujoco_menagerie) as the simulator; a hand-written Newton backend as a secondary
A hardened harness: harness-owned task parameters, last-step state measurement, AST pre-flight checks
Tavily for runtime failure-mode research before planning (sourced and linked in the UI)
A frozen evaluation config, with a resumable, budget-capped runner

Results (held-out, run once)

Six held-out tasks, five repeats each, frozen config. On reach: single-shot 2/15, best-of-5 6/15, best-of-5 + repair 11/15, + library 15/15. Total cost $1.72. On pick-and-place we got 0/15 in every condition, and we report it rather than hide it.

Challenges and honest limits

The effective sample is 3 tasks per template, so the intervals are optimistic. Individual steps are not statistically separable at this size.
The library gain is transfer across rewordings of one template, not new tasks.
The library only learns from verified solves. No dev pick-and-place task was solved, so it had nothing to learn there.
Results use MuJoCo only. Isaac Sim was not used.