RoboRetriever

Title Screen
Initial ideas
Initial ideas
Plan of action
Computation
Assembling robot
Setup
Finished building and setup
Testing the machine learning model
OpenCV Facial recognition

RoboRetriever Inspiration

Given the opportunity to develop a robot with AI-powered computer vision, ultrasonic sensing, and drivetrain mechanisms, we wanted to create a unique robot that could improve peoples' lives. We instantly thought of Boston Dynamic's "Spot" robot dog, and this led us to think about service dogs and how we could help the visually impaired. We learned that service dogs, while extremely beneficial, only help around 5% of all blind people around the world, given their high cost of upwards up to $25,000. We wanted to make a more affordable solution while retaining the personal connection a service dog would have with their partner, breaking down the notion that robots can't provide a similar sense of connection.

What it does

This RoboRetriever is connected to a leash for the user to hold, and the robot guides the person to their desired location. This robot efficiently navigates with it's mecanum four-wheel drive past obstacles such as fire hydrants and cars to forge a safe path for the user to travel on. Just like a service dog, this robot is highly aware of its surroundings, panning it's headlike camera around constantly to notice objects and alerting its owner of what kind of object it senses. In addition, the robot recognizes faces, tilting its metallic head towards them and making a special noise to acknowledge the person it sees.

How we built it

Our robot incorporates a plethora of software to be able to execute these features. With the starter kit provided for us and the source code libraries, we learned to implement the ultrasonic sensors and LEDs to sense based on distance, the mecanum drive to strafe in any direction without the need to turn, and the servos to pan and tilt the camera to see more of its surroundings. We used OpenCV to access the camera and created a face-tracking algorithm to have the camera follow human faces as it treks along the path. Finally, we utilized Google's Teachable Machine, a machine learning model creation tool that uses TensorFlowLite, a lighter version of TensorFlow to create a model trained off of hundreds of images of objects representing objects in real life. We incorporated this model into the robot and combined it with OpenCV's camera to detect these objects and react by alerting its owner differently depending on the object detected.

Challenges we ran into

Because we had to understand the source code from scratch, we ran into many issues, mainly with understanding how to use the functions provided as the source code did not provide much documentation. We have also hardly used OpenCV or have done model training before, so many hours were spent in the documentation, testing out different methods and algorithms until we found ones that we wanted to use for our project. On the hardware side, we did not consistently check the hardware for issues, which led to us later realizing the the wheels were loose, causing us to believe it was a software error.

Accomplishments that we're proud of

We're very proud of how we were able to make a finished product by the end of our first hackathon. We thoroughly deliberated over many ideas and created a structured plan with short, achievable objectives before jumping into the technical aspects. While there were features we would've like to add that we were unable to incorporate due to time constraints, we feel very proud that we were able to learn and use these powerful technologies like OpenCV and Tensorflow that are used so widely in today's world. We hope to be able to employ our knowledge for challenges we take on in the future.

What we learned

Aside from learning about new technologies, we learned that asking for help is always okay. In the beginning, we attempted to work on this on our own, but we realized that there is only so much we can accomplish by ourselves. After asking for help from the mentors from John Deere as well as other fellow competitors, we understood that to push our knowledge further, collaborating with others can widen our perspectives and foster new ideas. We helped other teams and learned from teams at the same time. By embracing the concept of gracious professionalism and coopertition, we learned to work with others while striving for victory.

What's next for RoboRetriever

We plan to smooth out all of the inconsistencies we found in our code and document our code much better for future users. We hope that this project serves as an inspiration to others to build robots that solve problems in the real world.