Smooth Operator

Close up view of the two racers!

Inspiration

While exploring project ideas related to Formula 1, we were inspired by the popular “AI learns to walk” simulations (such as “Albert Learns to Walk”), where an agent gradually improves through reinforcement learning. This sparked the idea to create an AI agent that could learn how to drive on an F1 track, and then integrate it into an interactive game where a human player can race against the model.

What it does

Our project is a racing game where players can drive alongside a reinforcement learning model on the F1 Budapest track. The player navigates through checkpoints and aims to complete the race, while the AI agent attempts to learn and improve its driving behavior over time. Although the model is still in training, it demonstrates early-stage learning and progression.

How we built it

We developed the game using Python with Pygame to handle rendering, track visualization, and player controls. For the AI component, we used Stable-Baselines3 in combination with Gymnasium to design and train our reinforcement learning environment. The model learns by interacting with the track and receiving rewards or penalties based on its performance.

Challenges we ran into

One of the biggest challenges we faced was the time constraint inherent to a hackathon environment. Reinforcement learning is an iterative and time-intensive process, requiring continuous tuning of reward and punishment systems. Because of this, the model did not fully converge or complete the track within the available timeframe. Balancing meaningful rewards while preventing unintended behaviors was also a significant hurdle.

Accomplishments that we're proud of

We are especially proud that all core gameplay mechanics function smoothly, providing a solid and enjoyable player experience. Additionally, our reinforcement learning model successfully began learning how to navigate the track, showing clear signs of improvement despite limited training time.

What we learned

Through this project, our team gained hands-on experience with Pygame for game development and visualization, as well as practical knowledge of reinforcement learning concepts. We also learned how to structure training environments, tune reward systems, and integrate machine learning models into interactive applications.

What's next for Smooth Operator

Moving forward, we plan to continue training and refining our model so it can reliably complete the Budapest track. Our long-term goal is to develop a highly competitive AI that can outperform human players and generalize its driving skills across multiple F1 tracks. We also aim to enhance the game with additional features, improved physics, and more advanced AI behavior.