Project Stop Lights

Poster

Title

Our goal is to identify vehicles that roll through stop signs.

Who

John Finberg and Sameer Sinah

Introduction

On campus, there are many streets with stop signs. It is common to see cars creep up to intersections and then proceed to go through them without ever stopping. This frustrated my group, and we wanted to create an automated solution to keep track of this type of event.

That leads us to our implementation of a solution. We are building an image classification model that, when given a live stream of our intersection, will provide bounding boxes for our crosswalks. We will then have another model frame-by-frame detecting vehicles in the frame. We will keep track of these vehicles and ensure they stop before the identified crosswalks.

Related Work

While there has been extensive research on autonomous driving and crosswalk detection, no one has combined crosswalk detection and vehicle detection together.

The following GitHub repository is on vehicle detection and bounding. link Using the YOLOv5 model, it is possible to assign vehicles a unique identifier. We can then track the vehicles throughout the frame.

The following research paper is on crosswalk detection. link We used this paper to help us determine techniques for building our own crosswalk detection model.

Data

We found the following crosswalk dataset link We will use this to train our crosswalk detection model. The dataset contains around 20k images which are annotated. These images are generated from a game named Carla.

Methodology

We will be training a crosswalk model. We have chosen a RCNN model to identify our crosswalks and to bound them. We will create a custom sequential Tensorflow model and go through the steps of training it in a Jupyter notebook.

The next part of the process is the YOLOv5 model to recognize cars driving and to identify and track each car uniquely.

Once we have both components completed, we will put them together. The RCNN model will identify the bounds of the crosswalk, and the YOLOv5 model will detect where the cars are. We will write code to ensure the vehicles stop before the crosswalk. Given the position of the car and the crosswalk overtime, we can see if the car stopped in an appropriate location or at all. We use the bounding box size of the car to scale the stopping distance accurately.

Metrics

We plan on recording a series of videos of an intersection. Each of these videos will be labeled as having a car rolling through a stop or not having a car roll through a stop. We will then test our app by running it on the collection of videos. Our goal is to maximize the accuracy that we get from this test. In addition to testing our overall application accuracy, we will evaluate both of our models on how successful they are. The crosswalk model and vehicle model will be evaluated using an mAP.

Ethics

We are unaware of which countries our crosswalk dataset was taken in. This means that in different countries, our RCNN model could perform differently depending on if crosswalks in those countries are different from our dataset.

One point to be made is that we are not using deep learning to make predictions about what types of vehicles will stop or roll through stops. This would make wide-reaching assumptions that could desperately impact specific groups of people.