Hand Gesture Controlled Car

The impact of autonomous systems are limitless with some creativity and innovation. We plan on showcasing the benefits these systems can have and their impact, using our hand gesture controlled car.

Comment

Inspiration

As soon as our group agreed to return to the MasseyHacks Hackathon for another run, we knew we wanted to head into this hackathon with more ambition and efficiency. Our goal was to challenge ourselves to make a complex yet reliable hardware project. For the project idea, we decided early on that a hand gesture controlled car was our favourite hack idea. Each member of the team has had previous experience in the FIRST Robotics Competition where we programmed and designed a robot that can be controlled in different ways. So we chose the hand gesture control because we wanted to do something more interesting and innovative.

What it does

The Hand Gesture Controlled Car is a car that is controlled through different hand gestures on a webcam where each hand gesture determines the speed and direction of the car. For example, closing your right or left hand would make it move forward, while opening it would make it stop.

How we built it

We refined our build process to be more organized, and separated the team into different subgroups.

  • Hardware Assembly and Wiring - Responsible for assembling the car kit and wiring. We used an l298n H-Bridge to control the motors.
  • Hand Gesture Recognition programming - Responsible for integrating the hand gesture tracking and recognition code to Bluetooth
  • Car programming - Responsible for programming the car kit using Arduino IDE so that final integration can be done between the car and the hand gesture recognition
  • Final integration - Responsible for integrating all the systems for the final result

Challenges we ran into

  • Never used Bluetooth to connect from a microcontroller (ESP-32) to a vision system
  • Final integration of all the different systems in our Hack
  • Overcoming the faults with our hardware - After a long time of testing, we learned our battery was bad
  • Integrating the variable speed in time for the final submission

Accomplishments that we're proud of

  • Achieving hand tracking and hand gesture recognition early on
  • Properly diagnosing and fixing the hardware issues
  • Getting much further than we did last year

What we learned

  • Basics of Bluetooth, being able to connect from a python program to an ESP board which is applicable to other boards
  • Understanding of the theory and applications behind machine learning

What's next for Hand Gesture Controlled Car

  • Using Bluetooth in more projects using other boards
  • refining our code and increasing the car's capabilities as a side project outside of the hackathon, adding variable speed to the program
  • applying the hand gesture recognition into other systems

Built With

Share this project:

Updates