Inspiration
Quadruped robots can move through environments that are difficult for wheeled robots, but developing a stable walking controller for a custom robot is challenging. This project explores whether reinforcement learning can teach the Hengbot Sirius quadruped to stand, walk forward, and develop a stable gait.
What it does
The project trains Sirius inside NVIDIA Isaac Lab using reinforcement learning. The current objectives are:
- Maintain a stable standing posture.
- Learn controlled low-speed forward movement.
- Develop a faster and more stable walking gait.
- Transfer the trained policy from simulation to the real robot.
The controller operates 12 active joints across four legs. It must control forward velocity while limiting falls, body tilt, lateral drift, yaw rotation, foot sliding, and excessive joint movement.
How I built it
I created a custom simulation model of Sirius from its mechanical and inertial data. The robot uses a parallel-link leg structure, so its joints and closed-chain constraints required careful modelling and validation.
The training environment is built with NVIDIA Isaac Lab and the Unitree RL Lab framework. PPO is used to train a neural-network policy with thousands of parallel simulated robots.
The policy receives observations including body velocity, angular velocity, projected gravity, joint positions, joint velocities, previous actions, and motion commands. A curriculum gradually increases task difficulty instead of asking the robot to walk at full speed immediately.
Challenges
The main challenge is that Sirius has a different mechanical structure from common quadrupeds such as Unitree Go2. Existing locomotion configurations therefore cannot be transferred directly.
Other challenges include:
- Correctly modelling the parallel leg mechanism.
- Preventing joint and linkage deformation in simulation.
- Balancing contact forces between the four legs.
- Avoiding policies that stand safely but refuse to move.
- Controlling yaw and lateral drift during forward movement.
- Preparing for sim-to-real transfer with limited real-robot sensor data.
What I learned
This project showed that locomotion performance depends not only on reinforcement-learning parameters, but also on accurate robot modelling, contact physics, actuator limits, observations, reward design, and curriculum structure.
A policy with a high training reward is not necessarily a useful walking policy. Independent evaluation of speed, stability, contacts, drift, falls, and gait quality is essential.
What's next
The next steps are to achieve reliable low-speed walking, gradually increase the target speed, add simulation randomization, validate the policy against real Sirius sensor data, and finally deploy the trained controller to the physical robot.
Log in or sign up for Devpost to join the conversation.