safedrive

Inspiration

9 lives. 1000+ injured. Distracted driving harms too many people every single day. We created safedrive with a simple but powerful motivation: to save lives. Every day, countless accidents happen due to distracted driving, and we wanted to change that. Our inspiration came from a desire to make the roads safer for everyone. By using machine learning and computer vision, we developed a tool that rewards safe driving and helps reduce distractions. With safedrive, we aim to create safer roads and communities!

What it does

safedrive is an ML-powered system that rewards safe driving and penalizes distracted driving. The system employs computer vision to analyze video feeds from a camera installed in the car. ML algorithms detect signs of distraction, such as texting, drinking, or looking away from the road. The system sends real-time data to a web dashboard for monitoring and intervention, encouraging safer driving practices.

How we built it

Please feel free to reach out to us at keval.botadra@gmail.com, anand.madarapu05@gmail.com, vamshipagidi6@gmail.com, or krtinakkineni123@gmail.com if you have any questions regarding any of these next few parts.

Artificial Intelligence

This artificial intelligence was trained on a state farm distracted driving dataset which contains 10 categories and hundreds of images. We tried two models, one being our own CNN sequential model and the other being a pre-trained vgg16 model trained on imagenet. The accuracy on our constructed CNN was much higher than that of the pre-trained model (ours being at around 97% with 0.05 loss while the pre-trained model was around 70% accurate with a 0.7 loss.

Before creating and optimizing our model, we conducted some data analysis to familiarize ourselves with the data. We found a few statistics (as seen below):

Classifications

c0: normal driving

c1: texting - right

c2: talking on the phone - right

c3: texting - left

c4: talking on the phone - left

c5: operating the radio

c6: drinking

c7: reaching behind

c8: hair and makeup

c9: talking to passenger

One thing we noticed early on with the data was how each image was taken from this specific angle in the car that we would have to recreate. The image also had to be produced in landscape not portrait. We all had these issues when we tried doing tests with pictures we took on our phones. Here is an example of a bad picture that Keval took with his mom as the subject. Below that picture is the collection of images from the dataset so you can have a better reference of what our images were supposed to look like.

Again, here is an example of the actual images from the dataset:

For the model, we planned on using multiple CNN layers for a multitude of reasons:

Convolutional layers are good at capturing spatial features: CNN layers are designed to extract spatial features from images. This is done by applying filters to the input image, which helps the model identify edges, corners, and other features that can be used to distinguish between different classes of images.

Multiple CNN layers can capture increasingly complex features: By stacking multiple CNN layers, the model can learn to capture increasingly complex features. The first layer may learn simple features like edges, while later layers may learn more complex features like shapes and textures.

Pooling layers reduce spatial dimensions: Pooling layers are often used in between CNN layers to reduce the spatial dimensions of the feature maps. This helps reduce the number of parameters in the model, which can make training faster and reduce overfitting.

Fully connected layers learn high-level representations: Fully connected layers are often added after the CNN layers to learn high-level representations of the features learned by the CNN layers. These representations can then be used to make predictions about the class of the input image.

How the model was built:

We trained the model on the 17939 train images and propagated 40 photos for each sample over 6 epochs. We noticed some overfitting in the model as the accuracy and loss plateaus toward the end of the training, however since we only trained it on 6 epochs this was not significant to affecting our model. Here are our plots for accuracy and loss:

As mentioned earlier we did try to conduct some transfer learning by using a pre-built model, however our CNN was performing better, so there was no need to continue it at the time.

Device

Our device consists of three main components: the battery pack, the raspberry pi, and the webcam. The pi is running scripts that send images from the webcam every 5 seconds to the backend code. It uses opencv to handle the camera use and the requests library to make the POST requests. The script was very straightforward as you can see here:

Here is what our setup looked like in the car:

Backend

Our backend was written using Python and specifically the Flask framework. We use one endpoint that receives images, classifies them, and uploads it along with the date and classification to Firebase. Here's what our backend looked like (it was pretty straightforward and contained just a little bit of data):

In the future, we are planning to implement the account systems so that each device will be connected to a raspberry pi and everyone will have their own personal statistics.

Web Dashboard

Finally, our web dashboard was developed using Flutter. It’s a pretty straightforward app that pulls data off Firebase and compiles the data in a visually-appealing way. In the future, we plan to implement the incentives features that will allow users to win prizes and compete on a leaderboard. Here's what it looks like at the moment:

Challenges we ran into

We ran into a few challenges. Firstly, in the beginning, we had low accuracy on our ML model because we were using inconsistent data. We were pulling data from multiple different sources which was causing discrepancies. We countered this by using one specific dataset from Kaggle, that allowed us to train the model to be accurate but not overfitted. Secondly, we took an extended period of time to connect our Firebase to our Flutter project due to the specific features we were implementing.

One of our biggest struggles with the device was the framerate. Our initial goal was to send as many frames as possible per second to our database in order to increase our sample size when making predictions about the user's driving, however, the webcam via USB connected to the pi was running at one of two frames per second. In the end, we just cut our losses and switched to one frame every five seconds.

Lastly, another problem was time management. We took longer on things that should have been done quicker and could have implemented more features such as a profile or increased framerate. Overall, these challenges taught us a lot about coding, teamwork, and dividing work.

Accomplishments that we're proud of

We are extremely proud of our project, Safedrive. We have managed to develop a highly functional prototype of safedrive that can accurately detect distracted driving in real-time. The system's impressive performance displays a high accuracy while minimizing false positives. This is a really strong model that has lots of practical applications that can genuinely help people. We are just happy that we were able to create a project that was such a large-scale for us. It was two of our teammates (Krtin and Anand) first time making a flutter website, so we were proud of that. It was also our most difficult hardware hack, so toying around with the raspberry pi and making it work with the machine learning model was a little difficult at first, but we pulled through after hours of debugging.

What we learned

The safedrive project was a valuable experience that taught us important lessons. We learned that the quality of data used in machine learning models is crucial for their success. Data accuracy, completeness, and relevance are vital to producing accurate predictions and reducing false positives and negatives. Optimizing models for real-time performance is also critical for their effectiveness in preventing accidents. Nonetheless, the potential impact of these technologies on improving road safety is significant, and we are motivated to continue developing them. Overall, the safedrive project offered valuable insights into the challenges and opportunities of developing AI/ML-powered technologies for road safety that will inform future developments in this field. Beyond AI/ML, we dived deeper into the use of hardware, which has opened an entire world of newer and better projects that we plan to build in the future.

What's next for Safedrive

We have an exciting vision for the future of our technology. We plan to implement an incentive system with even the hopes of our own cryptocurrency, safecoins. This crypto will act as another way to promote well-being on the road. We hope that the incentive of winning prizes and competing on a leaderboard will reduce distracted driving!

Additionally, we plan to optimize our model to increase accuracy and minimize loss. Also, we have to increase the framerate to multiple frames per second as this would represent a real-time model and wouldn’t miss anything

Built With

• c++
• computer-vision
• convolutional-neural-network
• firebase
• flask
• flutter
• html
• javascript
• machine-learning
• python
• raspberry-pi