Nod

Inspiration. Robot arms could give people with tetraplegia or ALS back everyday independence, but few get one and most are hard to drive. Of 85 people with ALS who were eligible for an arm, 32 received one, after about 197 days (Spittel 2024). Opening cabinets and drawers with a manual joystick succeeded only 39% of the time (Sensors 2025), and switching joystick modes takes 17.4% of task time on a JACO arm (Herlant 2016). Nod replaces mode switching with goal-level choices.

What it does. A phone propped in front of the user tracks their face on the device. They raise their eyebrows to open a menu, or say what they want. Nemotron proposes up to three options. A deterministic validator throws out anything unsafe or impossible. The person turns their head to pick a card and holds a smile (about 0.7 seconds, calibrated per person, eyes open so they can watch the ring fill) to confirm. After a one-second countdown the arm does the task. Holding the jaw open, saying "stop", or pressing the button halts the arm at once, and that path never touches a model. Risky skills are shown as "Needs caregiver" and cannot be confirmed by the user alone.

How we built it.

• Phone app (PWA): MediaPipe face landmarks on the device, a gesture state machine, a per-person calibration (about 90 seconds), the cards and the stop latch.
• Gateway (FastAPI): session state machine, a safety fast path, the validator, a skill registry with measured success rates, and a recorder.
• Nemotron on Nebius Token Factory: nvidia/Nemotron-3_5-Lightning for the fast path (thinking off, 2.5 s timeout, then a deterministic menu) and nvidia/nemotron-3-super-120b-a12b for multi-step plans and recovery. Replies are structured JSON, checked against the enums of the skills that are feasible right now, then checked again by the validator.
• Speech: NVIDIA Parakeet TDT 0.6B v3 (int8) on the laptop CPU.
• Simulator: the real LIBERO-Goal scene in MuJoCo on a CPU, with a 3D presenter view, a HUD (state, e-stop latency, LLM latency) and a trace of every planner and validator decision.

Honest scope. The arm is simulated. In this submission a scripted waypoint controller moves it; we measured it on the real scene, 20 episodes per skill (17/20 to 20/20, and a three-skill snack chain 20/20 in one session). The GPU tier, with GR00T N1.7 as the learned actor, Cosmos-Reason2 as the verifier and a nightly Nebius Serverless Job, is implemented and tested against fakes, but it was not run for this submission, because we used only the free Token Factory credits. Nod is a research prototype, not a medical device. A real arm would need a hardware emergency stop that does not depend on a camera, a phone or a network.

NVIDIA and Nebius in the submitted build.

Where Token Factory helped. The whole language layer needs no GPU of ours: planning runs on Token Factory, so Nod runs on an ordinary laptop. [MEASURE: Lightning p50 and p95 latency, and the share of valid structured replies, from the smoke test]. [MEASURE: which structured-output mechanism Token Factory accepted: json_schema, tools or json_object]. The free credits covered all development. [MEASURE: tokens or dollars used, from the Token Factory usage page].

Challenges. Our first confirm gesture was a long blink. In our own testing the person could not tell whether it had registered, because their eyes were closed, so we replaced it with a held smile and a visible ring. The validator exists because a language model should not be trusted with a robot: it is a pure function we can test exhaustively (hundreds of automated tests).

What we learned. Calibration matters more than the model: people smile, talk and blink all the time, so every threshold is set per person from their own resting and deliberate signals.

What is next. Run GR00T N1.7 and Cosmos-Reason2 on Nebius GPUs (the live stack is built), test with occupational therapists and people who have tetraplegia, add "nudge" options for fine control, and put a hardware stop in the loop before any real arm.

Built With

• fastapi
• libero
• mujoco
• nemotron
• nvidia
• python