Thermal FLIR camera image-based controlled Robocat

a mobile robot that moves autonomously based on image processing script of real-time images from thermal FLIR camera

Comment

Inspiration

To go further ahead with autonomous mobile robots, my team and I tried using thermal imaging FLIR camera to take real-time thermal images of two different objects and apply some machine vision techniques to detect and segment the objects, find the orientation of each of them and their distances in order to make our mobile vehicle to autonomously chase after the other object.

What it does

The custom built mobile robot (named Robocat) receives commands generated from the image processing script that tracks the orientation of the robocat to the mouse robot it's supposed to capture. The image processing script will give out the angle to which the robocat should turn in order to line-up co-linearly to the mouse robot, then the robocat receives a secondary command to how far it should advance in a forward direction in order to capture the mouse robot. Our robocat has a custom made gripper arm that engages once the mouse is at a certain distance to itself.

How I built it

Used Solidworks to generate the parts for the gripper arm and the parts to interface our motors easily to the base and its shield part. Used bluetooth module HC-05 to communicate from the Teensy connected to the FLIR camera to Matlab in order to generate the commands to move the DC motors.

Challenges I ran into

To get Matlab receive and send out communication was harder than we thought for some reason. Image processing script did not work out as fully as we wanted to send out correct commands to the motors. We needed to average out the output commands for each frame as FLIR camera reads about 20 frames per each second. Another challenge was having a dependable sources of heat and cold objects that would be constantly detected.

Accomplishments that I'm proud of

The design of my robot, assembly and functionality of it. Current image processing works too, but not always.

What I learned

Our team and I learned how to debug for this sort of frame by frame image processing task. Through this project, I was able to apply some of the computer vision techniques I have learned in class into application.

What's next for Thermal FLIR camera image-based controlled Robocat

put in more stable thermal and cold objects for detection, and improve on our detection and control algorithm.

Built With

Share this project:

Updates