Creature Forge

When a robot falls, NVIDIA Nemotron investigates why, proposes a physics-based redesign, and MuJoCo tests whether it actually works. Powered by Nebius Token Factory.

Comment

Inspiration

When a robot falls, designing another robot is easy. Understanding why it failed is much harder.

Large language models can suggest convincing physical explanations, but a convincing explanation is not necessarily correct. In our first live investigation, the AI proposed reducing torso mass to improve stability. It sounded reasonable. When we tested the idea in MuJoCo, the robot still failed.

That failure inspired Creature Forge: an AI-powered engineering laboratory where every proposed repair must survive an independent physics test.

Our guiding principle is simple:

The AI proposes. Physics measures. A fixed rule decides.

What it does

Creature Forge brings NVIDIA Nemotron and MuJoCo together in an agentic investigation workflow.

A user can draw a two-legged robot in a browser-based Studio, challenge it to stand, recover from a push, or walk, and observe what happens.

When the robot fails, the system follows a structured process:

  1. Measure the failure. MuJoCo records physical evidence, including falls, contact forces, posture, joint behavior, and push resistance.
  2. Investigate with AI. NVIDIA Nemotron 3 Super, accessed through Nebius Token Factory, receives the measurements and proposes evidence-grounded hypotheses.
  3. Run diagnostic experiments. The agent can request up to three controlled physics probes. Creature Forge validates the requests, executes them in MuJoCo, and returns the measured results to Nemotron.
  4. Propose a redesign. Nemotron interprets the experimental evidence and suggests physical changes within predefined engineering limits.
  5. Verify independently. MuJoCo evaluates candidate repairs on held-out simulation seeds, alongside a size-matched random edit. A deterministic verification rule decides whether an improvement is supported.
  6. Accept the repair. A verified candidate can be accepted by the engineer as the robot's next version.

The AI cannot declare its own idea successful. Its hypotheses and proposals are tested independently.

One robot, from failure to verified repair

Our recorded end-to-end demonstration follows a robot that initially failed a 15 N push challenge on all eight test seeds (0/8 passed).

Nemotron investigated the measurements and requested diagnostic probes. The experiments showed that:

  • Doubling friction made the result worse.
  • Increasing motor torque by 50% produced no meaningful improvement.
  • Increasing foot length by 50% improved stability.

Using that evidence, Nemotron proposed a more conservative redesign: increase foot length by 35%.

MuJoCo selected this candidate through screening and verified it on 12 held-out seeds, with 12 paired wins and zero losses against the original design. The size-matched random edit did not achieve a verified improvement.

After the engineer accepted the redesign, the new robot version passed 8/8 push tests on a new seed panel.

The recorded video shows this complete investigation, including the actual application, measurements, model interaction, proposed redesign, and independent verification.

How we built it

AI and reasoning

We use NVIDIA Nemotron 3 Super (nvidia/nemotron-3-super-120b-a12b) through the Nebius Token Factory API.

The model receives structured evidence rather than an unrestricted description of the failure. Strict JSON-schema guided decoding constrains hypothesis identifiers, editable parameters, and evidence references.

The investigation uses two model calls when diagnostic probes are requested: first to investigate and request experiments, then to interpret their results and propose repairs.

Physics and verification

MuJoCo 3.15 handles simulation, diagnostic probes, robot challenges, and independent verification.

Our Python validation layer checks model outputs against allowed parameters and physical bounds. Invalid proposals are rejected rather than silently modified.

Verification uses held-out simulation seeds and predetermined decision rules. The model's confidence is not used to decide whether a repair worked.

Cloud infrastructure

Nebius Token Factory provides hosted Nemotron inference. Nebius AI Cloud Compute VMs were used for large batches of physics experiments and evaluation.

We implemented request and spending limits, inference logging, and VM lifecycle safeguards.

Interactive Studio

The application uses FastAPI, React, TypeScript, and three.js. Users can create robots, run physics challenges, inspect evidence, compare candidates, and accept verified redesigns.

Recorded research demonstrations preserve the original experimental records and distinguish genuine model calls from scripted or non-AI procedures.

Results and accomplishments

Beyond the individual robot demonstration, we conducted a pre-registered pilot with 12 AI investigations across six starter creatures and two replicates.

  • Nemotron-selected redesigns: verified in 6 of 12 investigations.
  • Size-matched random edits: verified in 1 of 12 investigations.
  • Exact one-sided McNemar test: p = 0.031.

This is encouraging evidence that AI-guided physical redesign can outperform random changes under the tested conditions.

However, the pilot is small, primarily involves push-recovery challenges, and did not establish successful AI walking redesigns. We report these limitations rather than generalizing beyond the evidence.

The repository also includes additional research on experiment selection, diagnosis, walking recovery, and strict verification. Positive, negative, and inconclusive findings are documented with their experimental protocols and supporting records.

Challenges we faced

Getting reliable structured output. Early model configurations sometimes returned empty answers or invalid identifiers. We calibrated Nemotron configurations and constrained identifiers through JSON-schema guided decoding.

Separating plausible explanations from real improvements. Some AI proposals sounded physically reasonable but failed in simulation. Independent testing became a core architectural requirement.

Building meaningful baselines. A successful AI repair alone does not demonstrate an AI advantage. We therefore used matched random edits and pre-registered evaluation procedures.

Reproducibility. Physics calculations can differ slightly across operating systems. We preserved experimental environments, seed assignments, recorded outputs, and provenance checks to support reproduction.

Cloud reliability. Long-running simulation studies required additional safeguards for compute spending, authentication expiration, and reliable VM shutdown.

What we learned

The most important lesson is that AI can be useful as an investigator without being trusted as the final judge.

Nemotron can interpret evidence, identify worthwhile experiments, and propose repairs. MuJoCo supplies the physical measurements needed to challenge those proposals.

We also learned that success in one setting does not guarantee general performance. Some additional research comparisons found parity with deterministic methods, while others favored conventional engineering procedures.

Creature Forge does not claim to have demonstrated autonomous learning from accumulated experience. That remains an open research direction.

What's next

Our next goal is to investigate whether the system can reliably reuse previously verified physical experience, with controlled comparisons against an identical system without memory.

Other directions include more complex robot geometries, richer three-dimensional environments, and broader validation.

Ultimately, we envision Creature Forge as a verification layer for AI-assisted physical engineering: an AI collaborator that does more than suggest a design, because its suggestions arrive with measurable physical evidence.

Reproduce and explore

Source code: https://github.com/mahdinass/Creature-Forge

Judge testing guide: https://github.com/mahdinass/Creature-Forge/blob/main/docs/submission/JUDGE_TESTING.md

The repository is public under Apache-2.0 and includes setup instructions, experimental protocols, evaluation results, and provenance documentation.

The recorded demonstration pages can be explored without an API key. Local MuJoCo challenges and verification also run without cloud access. Live Nemotron investigations require the evaluator's own Nebius Token Factory API key.

Creature Forge is an independent hackathon project built for the Physical AI track. It is a simulation-only prototype, with no hardware validation. AI coding assistance is disclosed in the repository.

Built With

Share this project:

Updates

Submission history