Concept and Summary

In this project, a robot that is able to follow a particular person and automatically avoid obstacles was designed. To accomplish this concept, a raspberry pi camera sensor is utilized to detect the target person and interact with the central unit, which is Raspberry Pi in this project. Raspberry Pi would control the robot to follow the target person's movement. To make this happen, image processing is needed to detect the target pattern and control system is designed to control the robot. Meanwhile, the robot should also avoid any potential obstacles that could block it and cause collision. To realize this idea, several ultrasonic sensors were connected to detect the obstacles and provide feedback to the Raspberry Pi.

Baseline and Reach Goals

Baseline goals included:

•Be able to coordinate Raspberry Pi and OpenCV to detect a unique target pattern developed by the team and perform image processing.

•Design and build the body structure of a mobile robot car, which will be the carrier in this project. The mechanical body should include motors and batteries to generate power. It should also be able to move and change its direction smoothly.

•Use the central unit Raspberry Pi to direct and control the carrier robot to move following the target pattern it has detected.

Reach goals included:

•Use Ultrasonic sensors to detect the static obstacles that can block the robot. Avoid these obstacles.

•Detect moving obstacles and let the robot avoid these obstacles.

•Let the robot be able to reach the pattern on a person's waist.

It is both important and challenging

The problem is how to design a robot which can both follow a person's movement and avoid obstacles that block its way. The reason why we are trying to design such a robot is because human-following robots are used in numerous fields nowadays together with the explosion of improving daily life services and manufacturing demands. It could be used to take care of human in hospitals, especially in the purpose of assisting senior people and children. It can also be utilized in carrying loads in industrial zones, libraries, hospitals and airports. It can also help people during shopping.

To accomplish all the project goals, the problems and challenges includes: building mechanical robot body, perform image processing using OpenCV, power on the motors and control motors to let the robot follow the person, coordinate ultrasonic sensors to detect obstacles and avoid obstacles.

Overall System Architecture

The overall system architecture is that a Raspberry Pi was chosen to be the central control unit to process all the data and send signals to different body components. A Pi camera was connected to the Raspberry Pi to detect the target QRcode. It not only detects the information hidden in QRcode but also detects the length and width of the QRcode profile. Raspberry Pi receives the signal and information from its camera and give instructions to DC motors to control the wheels of the robot to decide whether to move forward, move backward, or stop. It also decides the speed of motor based on the distance calculated by Raspberry Pi. Four ultrasonic sensors were also connected to the Raspberry Pi to detect the obstacles. Two of the ultrasonic sensors were put at the front of the robot and the other two were placed at the two sides of the robot. Finally, a mechanical robot body was designed to be the body carrier to hold all the components on it. The final pictures of the overall system architecture were shown at the top.

Hardware Effort

The mechanical body structure is the carrier of the system. It holds the Raspberry Pi, Pi Camera, breadboards, batteries, and ultrasonic sensors on the body. It was designed using SolidWorks and printed using 1/8 MDF at Penn's laser cutting lab. All of the components it carries are stabilized through glue, wire, or body connection.

Ultrasonic sensors were connected to Raspberry Pi with one pin of Trigger and one pin of Echo. Both Trigger pin and Echo pin are connected to the Raspberry Pi's GPIO pins. Trigger pin is to send signals from ultrasonic sensor and Echo pin is to receive signals back to ultrasonic sensors to calculate distance between sensor and obstacle. The power supply of ultrasonic sensors is from Raspberry Pi 5V pin, and ground pin is also connected to Raspberry Pi's gnd pin.

Motors have 5 pins: Vcc, gnd, PWM, Direction, and FG. The required power is 12V, which comes from two batteries. PWM of the motor comes from Raspberry Pi's signal to control the rotation speed of motor. When the Direction pin is connected to 5V, it moves forward and when it is connected to ground, motor moves backward. FG pin controls the feedback pulse.

One thing that is critically important is that all the components of the system: Raspberry Pi, motors, ultrasonic sensors, breadboards, batteries should connect to a common ground.

Software Effort

OpenCV is utilized to detect the QRcode. Because the design goal is to detect and follow only the target person, the QRcode on the person's clothes must be unique. Hence, the hidden information of the QRcode is decided to be the email address of a team member to ensure the message of QRcode is unique. The QRcode of the robot is like the key of the door; the target person must hold the key to unlock the robot. Hence, the biggest mistake of this project: following a wrong person could be avoided.

To move the robot, motors are used to drive wheels and batteries are used to power the motors. When the camera finds the target person with QRcode, it will start to follow the movement of the person because camera will send signals to Raspberry Pi to control the motors. Nevertheless, the robot should not always follow the person because when the person is too close to the robot, the robot should stop to avoid hitting the person. Hence, the distance between the person and robot should be calculated. This is also done using OpenCV. When Pi Camera detects the target pattern, the python program will calculate the length and width of the picture to figure out the distance. There are several threshold distance values: moving forward threshold distance, moving backward threshold distance, stop distance. When the distance between the person and robot is larger than the moving forward threshold value, the robot will move forward. When the robot is pretty close to the person, the robot will stop. When the person is stepping backward towards the robot, the robot will also move backward.

After the program has decided whether to move forward, move backward, or stop, next step is to decide how to do it. In this program, the way to accomplish the moving actions is through controlling the Direction pin and PWM pin of motors to control the speed and direction of robot. In particular, the voltage on Direction pin will change robot's direction and the change of duty cycle of PWM will change robot's speed. To make the robot move and rotate smoothly requires an accurate math calculation on the relationship between motor's duty cycle and picture's width and length.

To use ultrasonic sensors to avoid obstacles, distances between ultrasonic sensor and blocking obstacle must be calculated accurately. To filter out the noise collected by ultrasonic sensor, a group of five distance values are collected together and an average value is counted. The underlying principle is that ultrasonic sensor sends a pulse traveling at the speed of sound; when the pulse is blocked by the obstacle, it travels back and will be received by the ultrasonic sensor. Hence, the time interval will be measured and distance can be calculated by multiplying time and speed. When the distance between the robot and obstacle detected by ultrasonic sensor is smaller than minimum threshold distance, the robot knows it will be blocked by an obstacle and will turn either left or right to avoid it. If the obstacle is on its left, robot will turn right, move forward, and lets its ultrasonic sensor on its left to detect whether the obstacle can no longer blocks it. Then it will turn left and move forward and keep following the target person.

Performances evaluation

The completed project generally achieves all the goals that were set. It is capable of detecting the QRcode on the person to determine whether the target person is and then follow the target person. It is also capable of detecting blocking obstacles using ultrasonic sensors and Raspberry Pi will give instructions to motors to avoid obstacles.

Further improvement on this project could be made to work on how to make the robot move more smoothly. Currently, there is a tiny delay between the real movement and the movement picture captured by Pi Camera, which causes a delay to the entire system. Perhaps a better hardware can be used to implement the project to make the robot move more smoothly and reach better performance.

Why is this project awesome?

This project is awesome because for a robot that performs autonomously, the communication between the person and the robot is the most important factor. And this research has a non-trivial involvement in the development of such robots. A robot that functions fully autonomously should not only complete the jobs that are desired of them but also somehow establish a connection between themselves and the person operating them. A lot of research has been done of these kinds of robot and a lot of work still needs to be done. In order for a robot to communicate and interact with the person, it should also be capable of following that particular person. Based on this point of view, this project is developed. Meanwhile, there should be a capacity in the robot to get information from the surroundings while pursuing the required object. So we fabricated a robot that not only tracks the target but also moves towards it while doing the tracking.

Share this project: