Find My Stuff Bot

This robot is here to find your stuff. Ask it what you are looking for and it will do the searching for you.

Comment

Inspiration

I have been dreaming about building a robot for some time. When I saw this challenge and noticed the GluonCV (recognize objects) model was part of it, I knew what I had to do.

What it does

The goal was to be able to say or text to the robot things like "find my guitar". And it would drive around the home, mapping its environment and using the GluonCV model to find the requested item.

In practice, I didn't even get close to that goal (see challenges).

So what it does right now: When started, the robot will expose a web page that shows his view on the world. Below that, are buttons to move the robot around. Also on the control page is a "identify" button that will snap a picture and send that to Sagemaker for identification. The resulting items are shown on the screen.

How I built it

The robot itself is built on top of a chasis I had bought some time ago, but never actually used. Its main "brains" are a Raspberry Pi 3 that is connected to AWS GreenGrass and AWS RoboMaker.

To move, there is a simple motor controller and to look around, it has the standard Raspberry Pi camera inside a 3D printed enclosure.

The software it runs is based on ROS (robot operating system), which is the thing that AWS RoboMaker uses.

The web interface uses Flask and React.

Challenges I ran into

There were many challenges. I'm an ok software engineer, but when it comes to robotics, I was hopelessly out of my league:

  • The battery for the motors is hopelessly underpowered. It can only run for about 5 minutes. However, a proper solution is pretty tricky (LiPo batteries have a tendency to explode if not treated well). This is the first thing I will be replacing.
  • The robot has two motors. One for the left track and one for the right. Because of that, it currently is not able to correctly move perfectly forward. To fix that, I would need to put encoders on the motors, measure and compensate the difference. Again, it's on my todo list.
  • For proper automatic mapping, the robot needs more sensors. Naively, I thought there would be some magic to fix all of this :)

Accomplishments that I'm proud of

What is there works! I know this sounds silly, but through RoboMaker, I can manage my "fleet" of robots (just one really) and feel like a mighty person from the future.

What I learned

  1. Getting up and running with AWS SageMaker and pre-built models was incredibly easy. I had the GluonCV network running in no time with surprisingly good results (and sometimes very funny ones too)

  2. Robotics is hard. There's so much fuzziness when it comes to working with "real things" that I'm not used to when writing software.

  3. Robotics is a lot of fun!

What's next for Find My Stuff Bot

I think the challenges section sums it up. I first need to make the robot more capable before I can tackle the actual problem I wanted to fix. So in order of importance: Battery, driving straight, sensors.

Built With

Share this project:

Updates