Laki2

Drones are becoming an integral part of our world, and we want to help make them more autonomous.

Comment

Influence

Small aerial vehicles are becoming more and more prevalent in our society. Not only are they being used more by hobbyists, but also commercially and industrially for agriculture, public services, film, military, etc.

What It Is

Our project was in collaboration with the UCF Robotics Club. The club is currently designing a large multi-rotor competition vehicle. For one part of the competition, the drone has to search an area and find ground markers, then classify those markers based on their shape, shape color, character, and letter color.

The difficulty comes from the fact that the images taken will be from a camera array on a vehicle in flight roughly 120' above ground, attempting to locate objects no more than 2' x 2'. Since it is in flight, all processing has to be done on-board the vehicle, and it will not have an internet connection. The effective image taken by the vehicle is roughly 3000 x 12000 pixels, with each ground marker covering at most a 160 x 160 pixel area. The flight causes significant blur and other noise to the object, and due to the nature of the task, both the shape and letter will be randomly oriented in the image.

Why It Does It

While this specific example is basic, the implications are massive. Similarly sized vehicles have been implemented in third world countries to deliver medical supplies when vehicle access is unavailable or impossible. Typically large centers deploy many of vehicles, and do not have the proper staff to manually fly all the vehicles simultaneously. Instead, the hospitals use large ground markers (combined with GPS), so that the drone can find the proper landing zone.

The same principal of autonomously locating ground markers could be applied to disaster relief services. Areas inaccessible via ground vehicles would be available to aerial vehicles. The software developed by the team would allow for less required human monitoring and allow for a more efficient use of resources.

In additional to supply transportation, industries like agriculture and safety inspections use aerial vehicles and ground marker to survey areas and stitch together images using the markers as a ground truth reference. There are also obvious military applications.

How It Was Done

The project had lots of different parts, but also lots of eager members. In addition to classifying the shape, shape color, letter, and letter color, the program must take the 3000 x 12000 pixel image and reduce it to a region of interest roughly 400 x 400, take that region of interest and segment the shape with a bounding box around the shape roughly 160 x 160 pixel, and segment further to the letter with a bounding box roughly 50 x 50 pixel.

Lots of different approaches were used to tackle the different tasks at hand.

For the initial region of interest generator, edge detection combined with contour finding proved to be a good method. Another solution was to look for rare colors on the mostly green background in a histogram as a blue or red object should stand out like a sore thumb. A third algorithm explored was using a keypoint method like SIFT or SURF combined with some clustering algorithm to create regions.

For the shape detection and classification, the team retrained a few popular CNN's using Keras. The CNN had an accuracy of ~92% at the end of the training. Additionally, more traditional methods like edge detection with contour detection and Hough transform line detection were also explored.

For the color classifications, histogram comparisons were explored and worked well.

Challenges

Most of the people working on the project were new to computer vision, new the project, or new to python. Catching everyone up to speed and getting everyone something meaningful to work on was difficult. However, being able to work as a team was a rewarding and fun experience.

What We learned

Prior to the competition, no one had experience training or using CNN's. Many of the members had never used opencv, and some had never touched python. Everyone learned something, and everyone got out what they put in. Overall it was an amazing experience, and everyone who isn't graduating plans to attend Knight Hacks next year.

What's next for Laki2

Up next, the team is going to continue working on the project with the Robotics Club. Most o This event was a great way to meet new people, and have the team members learn new things.

We want to give a special thanks to all the Tech Knights officers and members who planned and staffed at the event, as well as MLH and the sponsors who made it possible :)

Built With

Share this project:

Updates