Our submission for the 2026 Toyota Innovation Challenge held at IDEAs Clinic, University of Waterloo. Built over the course of two days, we focused our solution on safety when working with cobots, which got an honourable mention! Thank you to my team members Julian Chen, Baran Mangoli, Evy-Ralph Mwendalimbi Kalunga, and Chakib Belghait for being such fun and hardworking people!
Gestures - Feature 1
Uses the Dobot Magician to pick and place, but with added gestures using Mediapipe (peace sign for pause, thumb for resume) for safety during collaboration between the human and the robot arm. For the pick and place, a camera is used to detect an object and plate using HSV values and masks. Their coordinates are then transformed to world coordinates that the robot arm travels to. The Z coordinates are fixed.
Hand Avoidance - Feature 2
Dobot Magician pushes away from approaching hand, based on the hand's actual coordinates, velocity, and predicted coordinates, which are obtained from the camera feed. The hand is recognized using Mediapipe. The code isn't refined, expect it to work with some delay and inaccuracy.
You can learn more about our code and presentation using the GitHub link!
Log in or sign up for Devpost to join the conversation.