RoboPipe

• 

  Goal

• 

  Features

• 

  Infrastructure

Inspiration

Robot-learning data rarely arrives in one clean, reusable format. A dataset can have images, states, and actions and still be unsafe to combine with another: joint order, units, control mode, coordinate frame, timestamps, and camera alignment are often left implicit. That gap turns data preparation into a collection of fragile, one-off conversion scripts. RoboPipe was built to make those assumptions visible and reviewable before data reaches a trainer.

What it does

RoboPipe converts explicitly profiled robotics datasets into a canonical episode format and into common training-data layouts. It reads LeRobot v2.1 and v3, RLDS/TFDS, generic HDF5/RoboMimic, WebDataset, and self-contained ROS2 MCAP recordings. It can write canonical audit JSON, LeRobot v3, RLDS, WebDataset, and RoboMimic-compatible HDF5.

The important constraint is deliberate: RoboPipe will not assign meaning to a vector just because its shape looks familiar. A profile must declare the action representation, joint order, frame, units, timing, and camera mapping. The output keeps source provenance and transform lineage so a dataset build can be checked later.

How we built it

The core is a Python package with format adapters, a named canonical episode model, profile-driven converters, a versioned transform pipeline, quality and review utilities, and a CLI. Optional integrations use PyArrow for Parquet, h5py for HDF5, TensorFlow/TFDS for RLDS, MCAP ROS2 support for recorded ROS messages, PyAV for media, and the LeRobot runtime for compatibility checks.

Rather than forcing every source into one anonymous state/action array, the pipeline preserves named observation and action channels. It then applies only declared operations—such as nearest-sample synchronization, joint-space retargeting, action chunking, normalization, and validation—while recording what happened.

Codex CLI with GPT-5.6 was part of the development loop. We used it to explore the repository, implement and debug adapters and conversion paths, expand tests, and improve the documentation and demo materials. The project owner set the architecture and semantic contracts, reviewed changes and test output, and made the final decisions about what RoboPipe should accept or reject.

Challenges we ran into

The hard part was preserving semantics across formats, not simply decoding files. Two six-element action arrays can represent different joint orders or entirely different control modes. Video introduces another version of the same problem: frame files can exist without proving that they line up with control timestamps.

We handled that by treating missing semantics as an error. Conversion profiles and target preflight checks make the required contracts explicit, and media, timing, action-limit, duplicate, and structural checks surface problems before an export is used for training.

Accomplishments that we're proud of

RoboPipe now has an end-to-end, auditable path from a profiled source episode to several training-data ecosystems. It supports numeric data as well as RGB and lossless depth camera workflows, including ROS2 MCAP decoding, camera-calibration extraction, synchronization, and atomically published multi-episode exports.

We are especially proud that the project refuses to make silent semantic guesses. A failed conversion is more useful than a plausible-looking dataset with incorrect actions or camera timing.

What we learned

The lesson was that robot-learning infrastructure needs strong data contracts. File formats are only the outer layer; action meaning, embodiment, time, and provenance determine whether data can be reused. Making those decisions explicit also makes it easier to inspect, reproduce, and challenge a dataset build.

What's next for RoboPipe

Next up are broader ROS message and image-transport support, interpolated synchronization, and Cartesian/kinematics-based retargeting. We also plan to extend model-specific export support. RoboPipe currently validates target requirements but intentionally leaves model-owned action tokenization and normalization implementations to the training stack.

Built With

• codex-cli
• h5py
• hdf5-/-robomimic
• hdf5/robomimic
• lerobot
• mcap-ros2-support
• pyarrow
• pyav
• python
• rlds-/-tfds
• rlds/-tfds
• tensorflow
• tensorflow/tfds
• webdataset