Ugly Duckling

Inspiration

We wanted to meld our passions for AI, robotics, and cinematography into one mega project which would appeal to each of us.

What it does

Using deep learning and a face detection technique known as HoG, the robot follows the subject. The subject has been identified in advance, and the algorithm has been trained using a dataset containing images of Pranav (the "Mama Goose"). The robot is the iRobot Create 2, which is in essence a Roomba minus the vacuum.

How we built it

1. We constructed a base out of a wood, composite, & cardboard, placed it on the iRobot, and mounted a video tripod stand to support the camera.
2. We made a cardboard cradle for the computer that runs the robot.

Challenges we ran into

1. There was minimal documentation on how to program the iRobot available to us.
2. We had the python code that communicated with the iRobot on one computer and the face recognition software on another, but were unable to unify the two since they had different OS's.
3. When the two were finally merged, the face recognition and the robot motion control wouldn't execute simultaneously. We finally had to use multi-threaded operations in order to run both at the same time.
4. The laptop we used wasn't fast enough to handle the face recognition processing (it only had a dual core i5). If we had an Nvidia GPU, the algorithm would've been 50-100 times faster. This was why we experienced latency, and drops in face recognition at a distance.

Accomplishments that we're proud of

1. The video captured on the robot is very stable, much to our delight! We were able to reduce the wobbling produced by the robot significantly
2. We did it!

What we learned

1. More about machine learning
2. Robotic control mechanisms

What's next for Ugly Duckling

1. Improving facial recognition
2. Supporting larger/more complex camera rigs
3. Increasing the speed and power of the robot

Built With

• face-recognition-module
• irobot-create-2
• machine-learning
• python