OCR Driven Automated Vehicle

An automated vehicle powered by Raspberry Pi, Arduino, and Google Cloud's OCR platform.

Comment

Inspiration

Coming from a desire to create a project driven by machine learning, we decided to create a robot that simulated the behavior and potential of fully automated vehicles that employ machine learning to make decisions. We wanted to show that machine learning could be employed on a small scale, in order to conceptually prove the usefulness and feasibility of self-driving cars, as well as other intelligent, automated robots.

What it does

Our main project is an autonomous, self-driving robot that uses text recognition to interpret driving instructions such as "Stop" or "Turn right" in order to follow a "road of instructions". Furthermore, we built a simple second robot to act as a potential collision object in order to help build the autonomous robot's collision avoidance program. The end goal for this project was to create a model of an intelligent and automated car that will be heavily prevalent in the future.

How we built it

The main robot was built using Raspberry Pi 3 and Arduino which powered ultrasonic sensors and servo motors, while the simpler robot was built using only Arduino and two DC motors. Python was the main language that was used in coding the main robot, but the Arduino language was used for both the main robot and the simpler robot as well. The Snappy XO robot kits were used to provide the physical structures of the robots. Google Cloud's OCR (Optical Character Recognition) platform was used to capture and interpret text found in real life images. By utilizing Python, we were able to interpret text using the OCR API and convert them into specific movements controlled by the Arduino.

Challenges we ran into

One of the main challenges with this project was simply building an efficient and organized structure for the robots that would allow us to house all sensors and microprocessors, while not sacrificing speed or mobility. This was our team's first time using Snappy XO, but we were able to quickly learn the kit and create two functional robots. We also had some difficulty in training the program to recognize text printed on pieces of paper as instructions, and effectively translating them into the proper movement commands. However, the biggest challenge we faced was the limitation of the hardware we had access to, as the Raspberry Pi had to do live image processing, all while sending requests to Google Cloud's API and while preparing the movements of the robot. Effectively managing the resources of the robot was difficult, and proved to be a major bottleneck.

Accomplishments that we're proud of

This was the first experience that most members of our team had with robotics, as well as with team programming. In addition, we were surprised at how well we were able to integrate the Google Cloud API into our platform, and how accurate it was in interpreting the meaning behind text. We are extremely proud of the work we were able to do over this extended period, and the things we were able to learn in fields that we previously had no experience in. This was also the first hackathon for many members of our team, and we are sure to take this experience with us in the future.

What we learned

As mentioned previously, many of our team members previously had no experience with robotics, which resulted in us leaning a lot about the intricacies of wiring and circuitry as well as the possibilities that microprocessors such as Arduino possess in terms of prototyping and invention. Our team also learned to really work together and handle each other's issues in a unified manner, which is significant because we did not know each other before the hackathon. However, most importantly, we learned the value of perseverance and determination, as we would not have made it as far as we had if we had not worked as a team.

What's next for OCR Driven Automated Vehicle

We would like to create a vehicle that is much more physically efficient in terms of organizational structure and resources. Furthermore, we would like to integrate scene detection to truly embrace the circumstances that one would face on the road. Improving the machine learning and sensors used to sense the environment would be a huge leap forward for this project, and would be a great proof of the concept of autonomous cars.

Built With

Share this project:

Updates