A real frame from the demo: the detector didn't know the kiwi and proposed Discard. Nemotron, on Nebius, sent it to the basket.
A Raspberry Pi with a camera. Its pins drive a servo and five lights. The agent hands over an action, so any actuator can take it.
Reflexes and reversible calls stay on the robot, 65 ms, offline too. Throwing an item away always waits for Nemotron.
The robot proposed Discard for a kiwi it didn't know, waited 37 s offline instead of acting, then followed Nemotron: basket.
70% sure, the robot put a tomato in the basket at once. Nemotron's review called it a vegetable, and a white light asked to fix it.
Latest 19 decisions: 16 made on the robot, 65 ms against 840 ms, $0.10 against $0.18 per 1,000, and it keeps working offline.
The agent journals and queues on the robot and syncs over HTTPS. Only hard cases reach Nemotron 3.5 Super VL on Nebius.
Declare the task and what each action's mistake costs; the agent does the rest. MIT licensed, live demo at demo.cerebellumlabs.dev.

Inspiration

Teams that run robots on recycling lines, produce grading tables and warehouse floors face the same question for every decision: who makes it? A robot can send every frame to a large vision-language model in the cloud: close to a second per decision, a bill for every call, and a robot that stops when the Wi-Fi drops. Or it can trust a small model on the device: fast, but nobody catches its mistakes, including the ones that can't be taken back. On a produce line that mistake is good fruit in the bin; in a warehouse, the wrong item shipped. Teams end up writing their own glue between the two, and that glue tends to break offline.

Not every decision deserves the same treatment. Stopping when a person walks in has to happen now. Dropping fruit into a basket can be fixed later. Throwing something into the bin can't. So we built a layer that routes each decision by what a mistake would cost.

What it does

Cerebellum is an open-source decision layer for robot fleets. An agent on each robot routes every decision between the robot's own model and NVIDIA's open Nemotron model in the cloud. One server keeps the journal for the whole fleet, shown live on a web dashboard. The developer describes the task and marks each action by how costly a mistake is:

Class	Example	Who decides	Without a network
Reflex	stop when a person is in view	the robot, at once	works
Reversible	put an item in the basket	the robot's local model; Nemotron reviews unsure calls later	works
Irreversible	throw an item away	only Nemotron, in the cloud	waits in the queue

$$ \text{decides}(a) = \begin{cases} \text{robot, at once} & a \in \text{reflex} \\ \text{robot; Nemotron reviews if } c \lt \theta & a \in \text{reversible} \\ \text{Nemotron only} & a \in \text{irreversible} \end{cases} $$

Here $c$ is the local model's confidence and $\theta$ is the review threshold set per action, 0.9 in the demo. A reversible call that comes with no confidence is reviewed too.

Nemotron never drives the motors. It picks among the robot's own options: it decides the actions that can't be undone and reviews the robot's unsure calls, so a large model's judgment goes where a mistake would cost the most.

When a review disagrees with the robot, the robot gets a correction on its next sync and fixes the consequences. Every decision lands in the journal with its camera frame: who decided, with which model, how sure, how fast and at what cost.

The demo robot is a Raspberry Pi with a camera that sorts fruit. Fruit goes in the basket, anything else is thrown out, and a person in view stops the robot at once, network or not. Its lights show what it is doing: yellow while it waits for Nemotron, blinking yellow when there is no network, and white when a correction asks the operator to take an item out of the basket and press the button. Video never leaves the robot, only single frames do.

Every problem above has a direct answer. Measured on the demo robot's latest 19 decisions (13 person stops, 3 items it put in the basket, 3 it proposed to throw away), with cloud-only figures estimated from its 3 real cloud answers:

Problem	How Cerebellum answers it	On the demo robot
A cloud call takes close to a second	The robot decides on its own model	a decision on the robot took 65 ms; a Nemotron call took 840 ms, and the robot waited 1.0–1.3 s for it
Every cloud call costs money	Only irreversible and unsure calls go to Nemotron	16 of 19 decided on the robot; $0.10 per 1,000 decisions instead of $0.18, reviews included
The robot stops when the Wi-Fi drops	Journal and queue on the robot, sync later	a stop made offline synced 34.6 s later; a kiwi's discard waited 37 s offline for Nemotron, which sent it to the basket
Small models make mistakes nobody catches	Nemotron reviews unsure calls, the robot gets corrections	Nemotron overruled 2 discards (both kiwis) and corrected 1 basket call (a tomato) after review
Some mistakes can't be undone	Irreversible actions wait for Nemotron	every discard was decided in the cloud
Every team writes its own glue	One open-source agent and server, a token per robot	one journal and one cloud bill for the fleet

Without the layer, the same run goes worse either way. With the detector alone, 3 of the 6 sorting calls would have been wrong: both kiwis in the bin and the tomato left in the basket. With the cloud deciding everything, the robot would wait about a second on every call, stops for a person included, pay $0.18 instead of $0.10 per 1,000 decisions, and stop when the Wi-Fi dropped. With Cerebellum no sorting call ended wrong: Nemotron sent both kiwis to the basket, and the tomato was fixed within 16 s.

Try it: the live dashboard is at demo.cerebellumlabs.dev. The README's Quick start runs the whole stack on one machine, with uv run fruit-sorter sim driving it through the same robot code, no hardware needed.

How we built it

Agent (Python, on the robot): robot code calls agent.decide() with the task, a frame and the local model's answer, and the call never blocks. It shrinks frames to 640×480 JPEG before upload. The journal and the queue live in SQLite, so a reboot or a power cut loses nothing. Decisions made offline sync once the network is back. The server never opens a connection to the robot. Anything it has to say rides on the reply to the robot's next request. That way a robot behind NAT or on a phone hotspot still works.
Server (Go, PostgreSQL): /decide asks the cloud model about irreversible actions, /sync takes everything else and returns corrections, and a background worker reviews the unsure decisions. Each robot signs in with its own token. The server ships as one container image for amd64 and arm64 and runs behind a Cloudflare Tunnel.
Dashboard (React, on Cloudflare Pages): live over WebSocket. The overview compares Cerebellum with sending every decision to the cloud, the journal lists every decision with its frame, and each decision opens up to the cloud model's raw replies and what they cost.
Demo robot: Raspberry Pi, USB camera, YOLOX-Nano for people and objects, five LEDs, a servo and a button, started by systemd at boot.
Protocol: the agent makes UUIDv7 ids. Every update carries a revision, so a late retry can't roll a decision back. A retried /decide gets the answer already in progress instead of asking the model again. Device tokens are stored only as SHA-256 hashes: a leaked database can't sign anyone in as a robot. Any robot that speaks the wire format in AGENT_API.md can join, in any language.
Tests and CI: the agent is tested against a fake server that follows the wire spec. The sorter's logic runs end to end in a made-up world, with no camera or pins. The server has Go tests, and GitHub Actions checks every part. Server 0.1.0 is released as an image for amd64 and arm64 with signed build provenance.

NVIDIA Nemotron on Nebius Token Factory

The cloud brain is Nemotron 3.5 Super VL, called through Token Factory's OpenAI-compatible API with the camera frame attached as a base64 image and the reply in JSON mode. Reasoning is switched per call through chat_template_kwargs (reasoning_effort: none does the same). With Token Factory we never had to host a vision-language model: one API key, pay per token, and about $0.0002 per decision in the cloud. We use the model in two ways:

Deciding irreversible actions, with reasoning off: under a second and about 40 output tokens per answer.
Reviewing the robot's unsure calls in the background, with reasoning on: 1–1.6 s and 200–470 tokens in the demo journal, and nobody waits for it.

The server records each call's tokens and cost from Token Factory's usage data. A review gives up after 3 failed calls, and the robot drops a live decision only after the server reports 3 times that it couldn't decide, never because the network is down. The endpoint and the model are server settings, so any OpenAI-compatible vision model can take Nemotron's place.

Challenges we ran into

Reasoning ate the answer. Nemotron thinks by default, and with our first 300-token cap no call got an answer: the whole budget went to thinking. Before JSON mode, a reply after reasoning could also break the JSON. We turned reasoning off for live decisions, kept it for reviews, raised the token caps and asked for replies in JSON mode.
Offline is not a failure. Our first draft of the protocol counted a timeout as a failed attempt, so the robot could give up on an irreversible decision just because the Wi-Fi dropped. Now no network means wait, however long, and only an answer from the server can end a decision.
The camera sees more than items. Shadows and an arm reaching in looked like new items. The detector knows the 80 COCO classes, and kiwi isn't one of them. The sorter now finds items by comparing the frame with the empty platform, ignores long thin shapes at the zone's edge, and handles an item swapped for another. Basket decisions under 0.9 confidence go to review.
The servo jittered while holding still. Detaching it at rest fixed that.

Accomplishments that we're proud of

A real robot that keeps working with the Wi-Fi off: reflexes and basket decisions go on, discards wait in the queue, and the journal syncs when the network comes back.
Nemotron saved fruit from the bin. Twice the detector couldn't name a kiwi and proposed throwing it away; both times Nemotron answered "The object is a kiwi, which is a fruit…" and sent it to the basket.
Corrections close the loop. The detector took a tomato for an orange at 70%, and the robot put it in the basket at once. Nemotron's review answered "The object is a tomato, which is considered a vegetable in the everyday sense", and the robot fixed it, all within 16 s.
A public live dashboard where every figure is computed from the real robot's decisions, down to the raw reply of each Nemotron call.
Agent, server, dashboard and a working robot in five days.

What we learned

A vision-language model doesn't need to sit on every decision to make a robot safer. Put it where mistakes are permanent, and let it review the rest.
Reasoning should be chosen per call. On a single frame it made each answer slower and several times longer: right for a review, wrong for a live decision.
Designing for no network shapes everything: journal first, sync later, and never let the server call the robot.

What's next for Cerebellum

A hosted fleet server on Nebius, so a team only installs the agent.
Reviews and evaluations as Nebius Serverless Jobs.
Fine-tuning the robots' local models on the hard cases the cloud has already reviewed.
More NVIDIA models, robots and tasks beyond sorting.