Inspiration

DRIVE — Dynamic Risk-Aware Intelligent Vehicle Engine

Inspiration

Highway merging looks simple when a person does it, but an autonomous vehicle has to make several decisions at once: watch nearby cars, maintain a safe speed, find an opening, merge smoothly, and avoid making passengers uncomfortable. I built DRIVE to explore how reinforcement learning could balance those competing goals instead of optimizing for only speed or only safety.

What it does

DRIVE trains an autonomous agent to merge into multi-vehicle highway traffic. The agent observes a custom 25-dimensional representation of the environment containing information such as vehicle positions, speeds, lane states, and surrounding traffic.

Its behavior is guided by a reward function that combines four objectives:

$$ R = w_sR_{\text{speed}} + w_mR_{\text{merge}} - w_cR_{\text{collision}} - w_jR_{\text{discomfort}} $$

This encourages the agent to complete the merge efficiently while avoiding collisions and sudden, uncomfortable movements.

How I built it

I developed the project in Python using highway-env to simulate traffic and Stable-Baselines3 to train a Proximal Policy Optimization (PPO) agent. I created the observation space, designed the reward system, configured different traffic scenarios, and built the training workflow.

I also integrated Weights & Biases to track five important metrics:

• Crash rate
• Merge success rate
• Episode reward
• Episode length
• Rolling success rate

To make the environment easier to trust, I created a nine-test pytest suite covering important safety and behavioral conditions, including collisions, successful merges, speed rewards, and edge cases. The suite achieved full branch coverage across the custom environment logic.

Challenges

The hardest part was reward shaping. Early versions of the agent could maximize one metric while behaving poorly overall—for example, driving too cautiously to avoid collisions or maintaining speed without completing the merge. Small changes to the reward weights could produce very different behavior, so I had to compare repeated simulation runs instead of judging the system from one successful episode.

Testing reinforcement-learning code was another challenge because model behavior is probabilistic. I separated deterministic environment logic from the learned policy, allowing me to test reward calculations and state transitions independently while using experiment metrics to evaluate the agent itself.

What I learned

DRIVE taught me that building an ML system involves much more than training a model. The quality of the observations, reward design, testing infrastructure, and performance metrics can matter just as much as the algorithm. I also learned how to turn an open-ended autonomy problem into smaller components that could be measured, tested, and improved one at a time.

Built With

• env
• machine-learning
• pytest
• python
• stable
• weights