Pirates vs Navy

Inspiration

I was inspired by the show One Piece where pirates fight the navy.

What it does

The human players controls a pirate ship. The human player can choose to shoot at the navy ship and try to avoid the cannonballs from the navy ship. Each ship has 10 hp so the ship that survives the longest wins. The AI can move forwards, backwards, turn, and shoot similar to a human.

How I built it

The game is built using pygame. I found free ship sprites online (pirate, navy). The AI is trained using reinforcement learning using the stable baselines3 library, with the game being wrapped in an open ai gym environment. I used the PPO model type since it supports multiple binary actions and a continuous action space. I had the model play against itself while training. The video demo is the google drive link in the try it out area.

Challenges I ran into

Stable baselines was in the middle of migrating from open ai gym to gymnasium so there were a lot of version errors and there were a lot of methods that were not supported. It also took me a long time to figure out how to make the ships go forwards and backwards at an angle since the center of the image was not exactly the center of the ship. It was also challenging to switch the sprite image while looking like the same angle as the previous image since each sprite image has a 90 degree offset to the previous image.

Accomplishments that I'm proud of

I think the AI plays pretty well for such a short training time especially for the first few seconds and can hit a lot of the shots. I also think the move animations look decent for only four sprite images.

What I learned

I learned how to make games with pygame. I also learned about making custom environments for reinforcement learning.

What's next for Pirates vs Navy

I can add different difficulties for the AI. I can also retrain the model since currently I trained the model with a short time for each round. This causes the AI to play well for the first part of the game but then it gets worse. I can also increase the training time in general. I can also add a limit for the cannonballs so that the player can't just spam cannonballs. I could also make the animations smoother and make the water have more texture.

Built With

• gym
• machine-learning
• openai
• pygame
• python
• stable-baselines3