Using Rotational Bounding Box for Zebra Crossing Detection

Project Poster

Inspiration

Nowadays, deep learning is more and more useful in multiple areas for benefitting people’s daily life. “What objects are where?”, people raised this question, and then create a specific task called object detection, which is an important computer vision task used to detect instances of visual objects of certain classes in digital images. Road semantics and maps annotation remains a popular topic in deep learning as self-driving continues to be one of the most researched topics. Self-driving cars often require precise annotation of maps to perceive its surroundings accurately. In this project, we want to utilize object detection with deep learning to create a map annotation tool. We implemented a two component model that would read aerial map images, classify their categories and pass appropriate images to the following object detection network for annotation.

What it does

In this project, we want to utilize object detection with deep learning to create a map annotation tool.

How we built it

We plan to do experiments on lots of traffic images that contain traffic lanes and try to use bounding boxes to correctly mark them. Most of the dataset only contains images that are from non-aerial view, which is hard to label the suitable bounding boxes since they are easily out of shape. Google Map API is a good tool to download city images from aerial views.
The first part of our map annotation model is a map reader that utilizes the Google Maps Static API. We have implemented a reader that would download images of specific locations given the latitude and longitude or download continuous grids of images given a start point and how many images are wanted. After downloading, the images are passed through a classifier to determine if this is an image of a city with roads we want to annotate. The classifier is a convolutional neural network with 13 convolution layers following the structure of the residual block in ResNet (He et al. 2015). We trained the classifier on randomly downloaded images of wildlands and cities from Google Map.
The basic YOLO model (You Only Look Once) applies a single forward pass neural network to the whole image. It can predict the bounding boxes and their class probabilities. It has 53 convolutional layers called Darknet-53. For distinguishing the lane from other stripe patterns, we can directly compare the experimental results with the actual results. For the bounding box, we can compare the angle and size of the bounding box with the actual bounding box we want. When implementing the loss function for our YoloV3 model. Firstly, we are using the loss function as only considering the confidence level of the arrays of predicted boxes with the true labels and computing the maximum iou(Intersection over Union). It raises the issue that the spatial information of bounding boxes is difficult to maintain and therefore poor in accuracy. Then, we add many factors such as center loss, weight and height loss, and class loss to keep the ious accurately calculated by using the different specific loss methods. Furthermore, we finally move on to add degree as another factor and perform the atan2 method combining with a scaled binary cross-entropy.
Our base target is first to try to distinguish the crosswalk from other stripe patterns and then can use the basic horizontal bounding box to fit the crosswalk no matter the rotating or scaling of the lane. The stretch goal is to perfectly fit the crosswalk.