antares

• 

  UI

Inspiration

Give a stock robot simulator the exact commands a real SO-101 arm received, and it closes the gripper almost three seconds early. A sim that wrong about grasp timing can't tell you whether a pick-and-place policy will work.

That's the sim-to-real gap. Robots learn in simulation, but the simulator models a generic arm, and every real arm is a little different: servos wear in, firmware adds lag, no two units behave the same. When a policy fails on the bench, nobody knows whether to blame the policy or the sim. Today people hand-tune the sim or randomize it and hope. We wanted to measure the gap instead, using recordings people already make.

What it does

Antares makes your simulation run like your real robot.

You give it a LeRobot recording of your arm. It replays the same commands in a MuJoCo sim, tunes the sim until it moves the way your arm did, and saves the result as a small "lock" file for that one arm.

On average, we were able to decrease the average joint angle error by more than 60%.

How we built it

A LeRobot dataset logs what the operator commanded and where each joint actually went, 30 to 50 times a second. Antares starts MuJoCo from the real starting pose, plays the same commands, and scores the gap over the five body joints:

$$ \text{gap} = \sqrt{\frac{1}{5T}\sum_{t=1}^{T}\sum_{j=1}^{5}\left(q^{\text{real}}{j,t} - q^{\text{sim}}{j,t}\right)^2} $$

CMA-ES searches the servo gain, damping, rotor inertia, friction and command delay to shrink that number on some episodes, and we report it on the rest. A fit takes about half a minute on a laptop CPU.

Our stack is Python, MuJoCo, CMA-ES, NumPy, PyArrow, FastAPI, and React with Three.js for the viewer.

Challenges we ran into

• Many locks fit equally well. Two fits could reach the same error with stiffness values 25x apart. We addressed this by judging every lock only by how well it replays recordings it never saw.
• The fit kept cranking stiffness way up. Adding gravity load to the model kept it in a sensible range and lowered the error on all four arm and seed pairs we tested.
• Finding the camera. To draw the sim over the real video, we had to work out where each camera stood from the footage alone. We project the sim arm into the image and search for the camera pose where its outline best matches the real arm's. It only works with a fixed camera.

Accomplishments that we're proud of

• Every one of the 58 public arms we tried got closer to its real recordings.
• Every lock is robot arm-specific, validating our initial hypothesis that each robot arm's behavior is unique. An arm scores about 1.7x worse on another arm's lock than on its own, so the fit is learning that unit, not just a better generic arm.
• You can watch calibration happen live. Press Calibrate in the viewer and the sim arm slides onto the real one in about 30 seconds.

What we learned

• In our fits, timing explained most of the gap. The real arm reacts a few frames after each command, and the stock sim assumes no delay.
• The data people already share is enough to calibrate an arm. No special test motions needed.

What's next for Antares

• Run it on a live arm or other robot setup and check that arm with the drift gate.
• Run a real LeRobot policy (ACT or SmolVLA) through the calibrated sim.
• Measure whether a policy picked in the calibrated sim actually succeeds more often on the real robot. That's the result that turns this into a true sim-to-real claim.

Built With

• cma-es
• fastapi
• hugging-face
• mujoco
• numpy
• opencv
• pyarrow
• pyav
• python
• pytorch
• react
• scipy
• three.js
• typescript
• uvicorn
• vite