SmartRoomba

Smart Roomba that follows you using facial recognition. Track your Smart Roomba using our dashboard.

Comment

What it does

Our SmartRoomba is a personal mess-cleaning buddy that follows you around using facial recognition/tracking. Using a webcam mounted on the Roomba, the SmartRoomba detects its user's face, and autonomously follows them. The SmartRoomba pushes all its data to a slick web dashboard, so you can monitor your Roomba's location, trajectory, and sensor status on one dashboard.

How I built it

We used Python to control the Roomba and perform facial recognition. Our facial recognition uses Microsoft Cognitive Services and OpenCV. Our dashboard is served using Flask, and prettified with plenty of D3 (for the trajectory map) and CSS.

Challenges I ran into

One of the biggest challenges is detecting and computing the turning angle of the Roomba. The Roomba's wheel encoders provided some very limited and noisy data, so we had to figure out how to accurately retrieve the Roomba's angle for our trajectory map. Another challenge was efficiently performing facial recognition and tracking -- we experimented with different ways of doing this. This is also our first time using D3, so we spent a lot of effort getting some the dashboard elements to look nice.

Accomplishments that I'm proud of

We're very proud of figuring out how to use both Microsoft Cognitive Services and OpenCV to perform facial recognition, and using this facial tracking to control the Roomba.

What I learned

We learned how to make the most of noisy data from sensors, to compute the Roomba's trajectory angle. We learned how to do facial recognition, and how to use Microsoft Cognitive Services. We used D3 for the first time.

What's next for SmartRoomba

We think the best feature to add next to SmartRoomba are voice commands. It'd be great for the SmartRoomba to not only follow you around, but also respond to your voice. "OK Roombert, don't forget to vacuum the living room at 3:30!"

Built With

Share this project:

Updates