Driven

Coordinated Autonomous Driving via Reinforcement Learning #SandiaLabs #Google

Comment

Inspiration

Like many others, we would love to reduce our time in traffic. With autonomous driving on the horizon, the potential for re-imagining vehicle transportation is enormous. Rather than having vehicles move independently, we decided to approach this task by imagining the cars as a network of vehicles coordinating with each other. This way, the cars' access to more data would increase passenger safety while opening doors to more efficient paths of transportation.

What it does

We utilized reinforcement learning to train a model that coordinates the movements of a collection of vehicles. Based on their relative positions to each other and their environment, the model learns to minimize time in traffic while preserving driver safety (and the laws of physics). The resulting model was able to effectively direct cars towards their destination while noticeably reducing their time in traffic compared to status quo. We designed the model to reflect day-to-day scenarios, accounting for driver safety, fairness, and generalized environments.

How we built it

Due to the large scope of our topic, we decided to focus our attention on 2 key factors concerning traffic: 1) driver safety and 2) areas of multi-directional car movement, like intersections. We noticed that while roads come in all shapes and sizes, there are many underlying patterns from which we could develop our algorithm, from fairness to optimal car distance to Newton’s laws. We decided to capture these observations with a reinforcement learning algorithm due to its ability to generalize and model complex behaviors. After iterating through multiple approaches, we found that our implementation of a finite-state Markov Decision Process model in TensorFlow performed the best.

In order to run simulations of the algorithm, we used a lightweight Python gaming framework called PyGame. Later, we moved to Unreal Engine because of its support for everything from collision detection to rendering scenes. Using CAD software, we were able to take 3D assets and generate entire maps for simulations. Unreal Engine was also modified to include experimental Python and Tensorflow support.

Challenges we ran into

Since our algorithm kept track of large amounts of data, we needed to find a platform that could efficiently run our model while minimizing the time we'd need to set up and link additional frameworks. After trying multiple approaches in Python, we decided that TensorFlow had the best support for these requirements while allowing us to capture the complexity of the model. Since our problem was fairly unexplored, we needed to gain a lower-level understanding of Tensorflow and learn approaches to integrate our model with graphics.

Accomplishments that we're proud of

Collaborating and sharing the project workload efficiently. Since our project included everything from design to machine learning, we're proud of our ability to effectively combine our skills to realize a working contribution to a novel idea in 24 hours. We're happy that we were able to work on an idea that we're passionate about while offering a unique approach towards helping society at large.

What we learned

We're glad to be able to gain valuable experience on working on large projects together and learning more about integrating frameworks, machine learning, design, and brainstorming novel ideas.

What's next for Driven.

This project allowed us to explore an up and coming idea that has the potential to transform the way we live. There are numerous opportunities for helping bring our idea closer to reality in the years to come, from exploring novel algorithms, creating more realistic simulations, to collaborating with others passionate about this field.

Built With

Share this project:

Updates