Navigating the Future: Predicting Human Behavior for AVs

Special thanks to our mentors, Sandeep Thalapanane and Sandip Sharan Senthil Kumar and Professor Ming Lin for hosting this project!

What it does

As the development of autonomous vehicles accelerates, ensuring their safety in environments shared with human drivers is increasingly critical. This project aims to enhance the safety of AVs by using a virtual reality (VR) driving simulator to replicate a variety of pre-crash scenarios, allowing us to capture real-world human driving behaviors in a controlled setting. By immersing participants in these scenarios, we aim to collect data on how human drivers react in high-risk situations, improving our understanding of driving dynamics. Our project leverages predictive analytics to analyze human driving behavior, enhancing the capabilities of AVs in mixed-traffic environments. Through this research, we aim to:

• Improve safety: Develop algorithms that enable AVs to anticipate and respond to unpredictable human actions, reducing the risk of accidents.
• Inform design: Provide insights to guide the development of safer autonomous driving systems.
• Advance technology: Integrate behavioral data into existing frameworks to contribute to the ongoing evolution of autonomous technology.

The presentation outlines the background, methodology, findings, and conclusions, providing an overview of the research on safety in autonomous driving.

How we built it

We built the driving simulator using a combination of software and hardware tools to create an immersive driving experience. Roadrunner by MATLAB provided a library of road presets to simulate various street scenarios, while Sumo enabled the development of a 2D simulation illustrating how cars interact with the environment. For scripting, we used Visual Studio Code with XML, and Unity served as the platform to bring the 2D data from Sumo and Roadrunner into a 3D street simulation. Google Colab and Python were used to analyze user data and for the physical setup, we incorporated a Meta Quest VR headset, Logitech controller steering wheel, and faux driving pedals to complete the driving simulation experience.

Challenges we ran into

We faced several challenges during the development process. Time management was a significant issue, as understanding the interfaces and functionalities of Roadrunner, SUMO, and Unity required a substantial time investment. Creating complex and realistic driving scenarios, especially nuanced pre-crash scenarios, was also time-intensive. Additionally, the process of collecting, analyzing, and interpreting large volumes of data from simulations added a lot of complexity. Finally, the iterative nature of refining scenarios based on preliminary results required repeated cycles of testing and adjustments, adding to the overall project timeline.

Accomplishments that we're proud of

Despite the limited time, we are proud to have quickly grasped the functionalities of VR software, allowing us to create an immersive environment with props, terrain, buildings, traffic lights, and more. Our team was also able to integrate multiple tools—such as Roadrunner, SUMO, Unity, and Google Colab—effectively, which allowed us to build a driving simulator that blends 2D and 3D perspectives. Furthermore, we deepened our understanding of the computer research process for autonomous vehicles, focusing on driving behavior patterns, collision scenarios, and best practices for VR simulations. We applied this knowledge to create realistic driving interactions and safety critical scenarios. By analyzing user interactions and data from the simulations, we gained valuable insights into scenario modeling for VR, making this project a unique intersection of technology, research, and VR design.

What we learned

We gained valuable insights into the intersection of technology and human behavior in autonomous vehicles. Working with Unity, we learned to create interactive 3D environments, while SUMO helped us simulate traffic and model realistic vehicle behaviors. Roadrunner enabled us to design detailed virtual environments for testing autonomous vehicles. Together, these tools deepened our understanding of the complexities of driver interactions and the impact of different personality types on driving behavior. We also explored the computer research process, focusing on driving patterns, collision scenarios, and best practices for VR simulations. This knowledge allowed us to design realistic driving interactions and analyze user data, highlighting the unique blend of technology, research, and VR design in our project.

What's next for Navigating the Future: Predicting Human Behavior for AVs

The next steps for the project include expanding scenario variability by incorporating a broader range of driving scenarios and environmental conditions to enhance data quality. Additionally, we might conduct longitudinal studies to observe changes in driver behavior over time as participants become more accustomed to AV driving. The project will also focus on optimizing predictive algorithms, using collected data to improve real-time accuracy in anticipating human driving behavior. Finally, results will be validated through testing in real-world environments to confirm simulation findings and enhance the practical relevance of the research.

Built With

• colab
• python
• roadrunner
• sumo
• unity
• visual-studio