Inspiration

The inspiration for bewithdoggy comes from my house's recent entry into COVID isolation protocols and simaltaneous desire to hang out with some dogs at the dog park. What if there was a way to both isolate safely and see what's going down at the dog park down the street?

What it does

bewithdoggy is an integrated hardware system that allows an autonomous RC car to follow dogs around at a dog park.

The system consists of a RC car that livestreams a camera feed of the dog park to Twitch.tv, a laptop that takes screen captures of the livestream and runs them through a computer vision based object detection algorithm. Then the results are sent over the network to a Raspberry Pi controlling servos that manipulate the lever and dial on the RC Car's controller.

How we built it

We used a YOLO classifier for object detection based on their dataset, and filtered the results to look for specifically dogs in images. This classifier was set up using the Python implementation of the open source OpenCV library that is very popular for computer vision related tasks like this one.

Once the location of the detected dog is found (most importantly the position of the dog relative to the size of the screen to determine car direction), this information is sent over wifi using the SCP protocol by modifying a text file on the target Raspberry Pi to say what direction the next movement should be. The implementation of SCP that we chose to use was from open source SSH/related protocols library Parakmiko.

Once a position is translated into a direction to travel, the Raspberry Pi activates servo motors attached to the RC Car's controller to pull the controls in a hands off and automated way.

Challenges we ran into

-The car really wanted to just spin in circles in most cases

-Object detection is really difficult when using a lower quality camera that's also moving

-We initially tried to use Bluetooth instead of SCP but we couldn't get our HC06 module to behave

-The dogs were scared of our project and wouldn't stop barking at it :(

-Running a hotspot at the dog park required a lot of prep to not have to drag all our computers/monitors/etc.

Accomplishments that we're proud of

-First time doing computer vision!

-Got the RC Car previously to do some hacker stuff with but hadn't used it yet, so really glad we got to put it to use

-We actually got a live stream from the dog park, and the car interacting with the dogs!

-Completely subverting the control system for the RC Car came together very quickly

What we learned

-Using a pretrained model for computer vision is not nearly as hard to implement as I thought

-File transfer over Wifi is very nice for talking to a Pi from a laptop

-The RC Car is going to be a very fun tool for us in the future, and it is super hackable

What's next for bewithdoggy

-Adding more ways for dogs to interact with the car, like a stick on a string, an arm that pets them, or a treat dispenser

-Higher quality streams

Built With

Share this project:

Updates