-
-
ReFORGE System Diagram
ReForge: Robots That Continuously Learn From Failure
Inspiration
Modern AI agents can improve through feedback and reinforcement learning, so I asked: why can't robots continuously learn from their own mistakes while operating in the world? My previous research, CAPE (ICRA 2024), explored corrective planning for robotic systems, but relied on assumptions such as predefined executable skills, known action preconditions and effects, and abstracted observations rather than continuous low-level control. NVIDIA's DreamGen (GR00T-Dreams) demonstrated how Cosmos-generated synthetic demonstrations can expand robotic capabilities across new behaviors and environments, inspiring me to combine its generative approach with CAPE's failure-aware feedback mechanisms. ReForge takes this further: instead of generating synthetic data primarily for diversity and generalization, it targets observed execution failures to build a persistent robotic system that learns to recover and progressively improves its own policy.
What it does
ReForge is a continual-learning framework that enables robots to execute long-horizon tasks, detect and recover from failures, and improve their low-level manipulation policies without human demonstrations during operation. It combines four capabilities: high-level task decomposition, real-time execution monitoring, immediate failure-aware recovery, and longer-term policy adaptation through synthetic corrective demonstrations. When a robot fails, ReForge preserves the valid portion of its execution, reasons about the failure, synthesizes a corrective continuation, and uses the resulting experience to fine-tune its underlying GR00T policy. This creates two complementary learning loops: a fast feedback loop that helps the robot recover now, and a slower adaptation loop that helps it avoid similar failures in the future.
How I built it
I built ReForge around NVIDIA GR00T N1.7-DROID, using Nemotron 3.5 Super VL through Nebius Token Factory to transform natural-language goals and multi-camera observations into structured, spatially grounded plans with explicit preconditions and expected effects. GR00T generates low-level action trajectories, while a sensor-driven heuristic monitor tracks collisions, contacts, gripper behavior, and motion failures; Cosmos3-Nano-Reasoner diagnoses failures and proposes corrective actions when necessary. For continual learning, Cosmos3-Nano-Policy-DROID generates 8D joint-action continuations, Panda forward kinematics converts them into compatible action representations, and Cosmos3-Nano Forward Dynamics synthesizes the corresponding visual trajectories, which are combined with recorded execution data in a LeRobot dataset. I then fine-tune GR00T's diffusion transformer using rank-32 LoRA adapters, periodically reloading the adapted policy while preserving the active simulation state so learning and execution remain part of the same continuous process.}
Challenges I ran into
The hardest challenge was integrating several independently developed robotics systems into one consistent execution and learning pipeline: ManiSkill and Isaac/RoboLab differed in simulation interfaces, while GR00T and Cosmos required different camera layouts, action dimensions, and temporal horizons. Building reliable failure monitoring required extracting and calibrating contact, gripper, joint, and motion signals from simulation physics, while keeping these heuristics separate from task-specific success conditions. Synthetic demonstrations introduced another difficult alignment problem: Cosmos DROID produces 8D joint actions, Cosmos Forward Dynamics consumes 10D end-effector actions, and GR00T trains on 17D robot labels, all of which must remain synchronized with the corresponding visual observations. Finally, enabling genuine online adaptation required resolving data-format inconsistencies, replay-buffer indexing, checkpoint loading, and policy-server updates without accidentally resetting the robot's world between training cycles.
Accomplishments that I'm proud of
I'm particularly proud of building an end-to-end architecture that connects multimodal planning, closed-loop robotic execution, failure diagnosis, synthetic trajectory generation, and continual policy fine-tuning within one system. I developed challenging evaluation environments spanning shape-sensitive assembly, color-based cube stacking with distractors, and semantic object sorting, alongside a shared rollout and evaluation framework for comparing six increasingly capable system configurations. What makes this especially meaningful is that I built ReForge independently, having discovered the hackathon only a few weeks before the deadline, and brought together research ideas, simulation infrastructure, foundation models, and cloud GPU training into a single coherent project. Beyond any individual benchmark result, the accomplishment I'm most proud of is turning the idea of a robot that learns from its failures into a concrete, testable system.
What I learned
My biggest realization was that continual robot learning requires continuity of experience: if every failure or fine-tuning update resets the world, the system is effectively performing repeated offline training rather than adapting within an ongoing task. I also learned that failure recovery should be treated as a central control mechanism, not an exceptional fallback, because long-horizon manipulation depends on recognizing and correcting mistakes before they cascade into task failure. Perhaps most surprisingly, integrating foundation models trained for the same Franka robot was less about model compatibility and more about reconciling incompatible observation layouts, action semantics, control frequencies, and training conventions. These lessons reinforced my belief that the next step toward more autonomous robots is not simply larger models, but systems that can reason about, learn from, and retain their own experiences.
What's next for ReForge
My next goal is to extend ReForge beyond tabletop manipulation to more challenging long-horizon tasks involving tool use, articulated objects, and semantic reasoning, systematically measuring how well its corrective learning transfers across task families. I also want to make the synthetic-data generator itself adaptive, allowing its predicted future trajectories to improve from discrepancies between generated and observed outcomes rather than treating Cosmos as a permanently fixed model. Longer term, I envision deploying ReForge on physical robots, where continual adaptation must operate under real sensor noise, contact uncertainty, and safety constraints. The broader vision is a persistent robotic learning system that doesn't just execute tasks, but becomes more reliable through every interaction with its environment.
Log in or sign up for Devpost to join the conversation.