We wanted to do a super fun and really technically challenging project, so we decided to do something combining our passions: Computer Vision, Electrical Engineering, and Controls. We decided to make a nerf gun turret that recognizes faces, points in the direction of the recognized face, and shoots.

What it does

It recognizes a face, and points the turret in their direction, then shoots nerf darts at them.

How we built it

The turret has an OpenCV facial detection program running on a Mate 10 Pro. The Mate sends the coordinates of any faces found to a raspberry pi which reads the input from the Android Developer Bridge and forwards the data over a serial port to the Arduino which runs the controls for aiming and firing the turret.

Challenges we ran into

The biggest challenge we ran into was the lack of electrical supplies. Ideally, we would use a NEMA 23 or NEMA 34 stepper motor to drive our turret's rotation, but we could only get a NEMA 17. This caused our turret to be underpowered when it turned and caused very jerky motion control. We can fix this issue by simply upgrading the stepper.

Accomplishments that we're proud of

We are extremely pleased with our vision system. We got a full face detection algorithm working on an Android phone. We are really proud of our algorithm too because we achieved 250ms latency from when the image is taken to when the coordinates are sent to the RPI. Additionally, we are really happy that we got the full coordinate communication system working; we got the Mate to communicate smoothly with the Raspberry Pi, and then we managed to get the Pi talking to the Arduino, creating a pipeline of coordinate information. We are also really proud of how well our shooting logic works. The nerf gun's sequence shooting was well refined and worked very smoothly.

What we learned

We learned how to write OpenCV code for an Android phone. We also learned that we should plan our electrical components ahead of time, and be more proactive about making sure we have the exact parts we need, and that the parts are capable of doing what we need.

What's next for Nerf Turret

We'd love to upgrade the rotational motor so we can implement PID control on the rotational motor. This would enable us to get really precise shooting and take full advantage of our low latency vision algorithm.

Share this project: