Inspiration

I wanted to beat my friends playing Brotato in the living room

What it does

it is a highly parallel, very fast reinforcement learning environment to train an ai model to play brotato with Pufferlib 5.

How we built it

My agents and I decompiled and reverse engineered brotato, stripped the godot game engine from it so it only required highly parallelizable pure CPP and CUDA to run (basically rewriting the game) to achieve 5M steps per second with training the reinforcement learning policy with a consumer GPU.

Challenges we ran into

Ensuring the RL env matched brotato exactly (an ai model learns the environment so it must be as accurate as possible so the model is not disoriented when it plays the "actual" game. Built a game "mod" that but the policy into the the actual game (the binary that the game makers ships (with godot game engine)) to evaluate the real policy performance, so I can edit the env to match the game perfectly and also reward/punish my agent optimally.

Accomplishments that we're proud of

It works haha It achieves 2M SPS right now, using only 1.5mib of data for gaem/env so I have can run 4k envs parallel to tarin. Created a Brotato game "mod" to put the policy in the game to evaluate and find bugs, first in the sim/env and second in the policy

What we learned

You must learn the environment. Simulate it as fast as possible. Parallelize everything. Use the game as an eval for the policy and for the sim, make sure everything matches. I paralelizes the physics solver and contact solver with cuda, speeding things up from 1M SPS to 2M SPS. Right now when training 87% of time is taken by the env. it is memory bandwidth constraiined due to the massive amount of data to move around.

What's next for Brotato Edgerunner

  1. make the sim match the game exaclty (bit-exact)
  2. 5M SPS/get game state to 200KB/parallelize even more
  3. Train policy that defeats endless mode
  4. use other faster physics and contact solve methods to see if the policy still transfers to the game(sim2real)

Built With

Share this project:

Updates

Submission history