Tower-bot: Towers of Hanoi Solver

• 

  RGB camera images from one of the top-down cameras and frontal cameras on the left and segmented mask for each camera image on the right

• 

  Robot picking up the first ring from the peg

• 

  Sample trajectory from one pole to another. The cameras were later moved out of the way

• 

  Iiwa partway through its trajectory after dropping the first link

Inspiration

We were inspired by the challenge of robotic manipulation in assembly lines and industrial automation. The Tower of Hanoi problem serves as a great testbed for grasping, motion planning, and perception, all of which are crucial in real-world robotics applications. We wanted to see if we could get a robotic arm to autonomously solve this problem using perception and inverse kinematics.

What it does

Towerbot attempts to solve the Tower of Hanoi problem by perceiving the environment with RGB-D cameras, identifying the disks, and executing planned motions to move the disks between rods. The system uses an antipodal grasping strategy to pick up the disks and a motion planner to ensure smooth, collision-free movements.

How we built it

We used RGB-D cameras for perception and segmented the scene using color masks to detect the disks. The robot’s grasping was implemented using an antipodal grasping method, and the movement was planned using inverse kinematics with Drake. The Towers of Hanoi solver guided the sequence of moves, and we fine-tuned the trajectory to prevent collisions.

Challenges we ran into

One of the biggest challenges was accuracy in placing the rings. The robot sometimes misaligned the disks when dropping them, making it difficult to regrasp them later. Additionally, the inverse kinematics solver occasionally failed to find solutions, requiring us to implement intermediate poses to improve reliability.

Accomplishments that we're proud of

We successfully integrated perception, grasping, and motion planning into a functional system that can autonomously pick up and move disks. The system works consistently for the first few moves and demonstrates key robotic manipulation concepts in a structured problem-solving scenario.

What we learned

We learned a lot about robotic perception, grasping techniques, and trajectory planning. Debugging inverse kinematics failures and dealing with inaccuracies in physical object placement were key takeaways. We also gained experience in optimizing motion planning to reduce collisions and improve grasping stability.

What's next for Towerbot

To improve Towerbot, we need to refine the placement strategy, possibly by using a segmentation model to detect the pole position and adjust before placing the ring. Testing more complex grasping strategies and implementing better trajectory smoothing could also improve performance. We’d also love to expand this system to handle more complex real-world assembly tasks!

Built With

• drake
• meshcat
• python