Robot Learning for Keyhole Insertion

Inspiration

Many Industry tasks require high precision due to low tolerances. The combination of custom gripper engineering and State-of-the-Art imitation learning offers a powerful tool to quickly set up and learn complex manipulation tasks. We conceptualize, build, and demonstrate a working prototype within 36 hours.

What it does

Picks up a peg with a custom gripper and inserts it into a low-tolerance keyhole.

How we built it

After assembling the LeRobot SO-101, we played around with the teleoperation feature to explore what it might be capable of. After some deliberation, we chose an ambitious, low-tolerance key insert task. We designed and 3D-Printed a custom gripper and workpieces. Then we gathered demonstrations of a human completing the task, which we used to train our model.

Challenges we ran into

Robot Assembly: The Motors were given were not quite the same as the ones in the official tutorial, so we spent quite some time deliberating which motor is placed where to optimize stability and accuracy.

Training Time: In a 36h setting, training time becomes a significant factor. We had to plan around this limitation, by parallelizing training alongside other tasks, like additional data collection, research and hardware improvements.

Camera Setup: A second camera improves training significantly. Due to the lack of resources as well as the time constraints, we used the webcam of one of our teammates as a permanent addition to the robot scene and stream the video via Ethernet to the main laptop running the code.

Accomplishments that we're proud of

We are proud to be able to demonstrate that our policy is able to complete the task fully end-to-end and on-policy, without any hard coded robot movements.

We measured a success rate of around 40%

What we learned

Scope and task management is key in such a short amount of time. Building the robot first to see what it's capable of gave us the ability to accurately gauge how complex we can make the task. We also learned that starting as simple as possible and adding complexity later can yield fast results, with the additional option of always being able to fall back.

We also learned it's important to be quite deliberate about who spends time on which project.

What's next for Robot Learning for Keyhole Insertion

We explored a lot of areas for improvements, such as adding force sensors for higher accuracy insertion. To take it to the next level, we'd love solving a real-world task (e.g. from a factory).

Built With

• act
• amazon-web-services
• huggingface
• imitation-learning
• python
• sagemaker