Inspiration
Singapore already uses robots to clean floors in public spaces. We wanted them to clean tables, counters and handrails too. These are harder: the robot must press hard enough to lift a dried stain without scratching the surface, and tell a stain from wood grain or steel shine. We also wanted the robot to check its own work. A robot that wipes a fixed route and reports "done" can leave stains behind.
What it does
CleanProof is a physical AI software system that enables various robot platforms to clean intelligently. Cosmos 3 finds each stain, Nemotron 3 Super plans the wipe and its force, the robot wipes, and Cosmos 3 checks every zone again until the surface is clean.
- Appropriate force. Cosmos 3 scans the whole surface zone by zone and identifies each stain. Nemotron 3 Super on Nebius Token Factory looks up the starting force for that stain on that surface in an updatable force database. A force controller on the robot holds that force while it wipes.
- Varied surfaces. Cosmos 3 compares every zone with the site's clean-surface reference, captured once at calibration, so wood grain, stone speckle and steel shine are not mistaken for dirt.
- Proven clean. After every wipe, Cosmos 3 checks every zone again. If part of a stain remains, Nemotron re-plans with more force, up to the surface's limit, and a guardrail approves every plan before the robot moves. A mark that will not come off is handed to staff. Every job ends with a report: before-and-after images of each zone that needed work, every decision and force trace, and an audit by Nemotron 3 Ultra.
Our software is what an operator uses on site, on Linux as a desktop app or in a browser:
- In a setup wizard, the operator picks the robot and connects to the robot computer over an encrypted link. The wizard checks every service (Nebius Token Factory, Cosmos 3 on the local GPU, the orchestrator) and calibrates the site once.
- The operator can pause, stop or E-stop a job at any time. A staff-check request can trigger a notification or a webhook.
- Settings cover job options, per-surface force limits, models, spending limit and robot connection.
- A log keeps every model call with its latency and cost.
CleanProof supports real-robot deployment:
- It installs on an on-site GPU workstation. An install script sets up the robot computer, and the two pair over an encrypted SSH link with a shared token.
- The robot computer runs only the skill server, the single process that can move the robot, and a camera service.
- UR arms connect through an RTDE adapter with staged commissioning (shadow, dry run, then full), and a person on site arms each session. Other robots connect through software adapters, from fixed arms to a quadruped with an arm.
How we built it
- GPU workstation (on site): the orchestrator (agent loop, guardrail, report) and Cosmos 3 Nano (
nvidia/Cosmos3-Nano), served with vLLM on a local NVIDIA GPU. - Cloud: Nemotron 3 Super (
nvidia/nemotron-3-super-120b-a12b) plans each wipe, and Nemotron 3 Ultra (nvidia/Nemotron-3-Ultra-550b-a55b) audits each job, both on Nebius Token Factory. All model calls go through one client. It logs backend, model, latency, tokens and cost, validates structured JSON output, and enforces a spending cap. - Robot computer: a FastAPI skill server runs whitelisted skills (survey, close-up, force-controlled wipe) with safety limits, plus a camera service. The GPU workstation reaches both through an SSH tunnel with a shared token.
- Software: a FastAPI back end and a single-page front end, shipped as a Linux desktop app (pywebview). Every button calls a real endpoint and is covered by tests.
- Simulation and evaluation: MuJoCo physics with a stain model (per-surface removal thresholds, layered dried stains) and robot models from MuJoCo Menagerie. We ran a randomized benchmark whose seeds were registered before the runs. The video's 3D shots are rendered in Blender Cycles from the logged MuJoCo states.
Results in simulation, 60 randomized jobs per method:
- Jobs finished correctly: CleanProof 97% (58/60) vs. 60% (36/60) for a fixed route wiped without checking.
- Marked "done" while still dirty: 0% (0/60) vs. 40% (24/60).
- Wipes per job: 1.9 vs. 3.3 and 6.7 for the fixed route with one and two passes.
- Surface wear per job (normal force × wiping distance): 2.60 N·m vs. 5.03 and 10.01. Matching the force causes 33% less wear than pressing at 15 N on every wipe, with the same 97% of jobs finished correctly.
- Nemotron decisions for all 60 jobs cost $0.099 in total. A decision takes 1.53 s at the median on Token Factory.
Challenges we ran into
- Telling stains from texture. To a pixel diff, wood grain, granite speckle and brushed steel all look like residue. Comparing each zone with a clean reference recorded at calibration removed most false alarms. Fusing Cosmos 3 with a classical color check cut missed residue from 3.4% to 2.0%.
- Escalating force safely. Early on, Nemotron sometimes repeated the same light wipe instead of pressing harder. We added a guardrail that checks every plan against the force database and the surface limit, requires more force after a failed wipe, and sends invalid plans back to Nemotron with the reason.
- Structured output. With reasoning on, Nemotron could use up its token budget before finishing the JSON. We turned reasoning off for planning and validate every response against a schema.
Accomplishments that we're proud of
- In 60 randomized simulated jobs, the closed loop did not once report a dirty surface as clean, and it used fewer wipes and caused less wear than wiping a fixed route.
- One software stack for many kinds of robots through software adapters, from fixed arms to a quadruped with an arm.
- Software that takes an operator from setup to daily jobs: a setup wizard, live jobs, staff checks, settings that the guardrail enforces, and a log of every model call with its cost.
What we learned
Checking after every wipe matters more than a better first guess. Cosmos sometimes reads a dried stain as a wet one, and the re-check catches it. We also moved hard rules out of the prompt and into code. The guardrail checks every plan, so a rule holds even when the model ignores it.
What's next for CleanProof
On-site trials with partner robots on real surfaces, calibrating the force database on real materials, and multi-position planning for tables larger than one reach.
Disclosure
- All robot footage in the video is simulation (MuJoCo physics), marked "Simulation" in the corner.
- The opening and the three chapters are rendered offline in Blender Cycles from the logged MuJoCo states. The software's camera view uses MuJoCo's renderer.
- Footage is time-compressed: robot motion inside a shot plays up to 9× faster than recorded, and the whole video is sped up 1.1×.
- All numbers above come from simulation.
- Narration is a synthetic voice (Kokoro-82M).
- Photo credits are in the repository README.
Built With
- blender
- fastapi
- javascript
- linux
- mujoco
- mujoco-menagerie
- nebius-token-factory
- nvidia-cosmos-3
- nvidia-nemotron-3-super
- nvidia-nemotron-3-ultra
- opencv
- pydantic
- python
- pywebview
- universal-robots
- ur-rtde
- vllm