Kero chess bot

Inspiration

This project idea comes from my personal interest in both reinforcement learning and the game of chess. It takes heavy inspiration from deepmind's paper about alpha zero, but also combines them with more classical chess programming concepts.

What it does

This is an artificial intelligence that plays chess. It uses the predictions of a neural network trained using reinforcement learning to orient a tree of possible moves.

I trained the network through self play on a total of 1200 games. The full algorithm being very slow and expensive to run, it is not trained the same way it plays an actual game: during training, the move selected is simply the one that has the highest predicted outcome according to the model.

To play an actual game, we combine what it learned with a treesearch algorithm. Every node in the tree of possible moves is also put into a priority queue, with a priority determined by a combinaison of how deep the node is and the predicted value of the position according to the model. Then we explore the tree in the priority queue order, which favorizes going deep into variations the model likes over ones it dislikes and disregarding a lot of moves that immediately look terrible to focus on the good parts of the treesearch.

Then, once the search is completed, we apply a minmax algorithm on the resulting tree to get the final move to play.

How we built it

This model was built mostly using keras' convolutional neural networks. For the frontend (webapp), I used gcloud and flask to make a website with python as a backend.

Challenges we ran into

The trickiest part of the project was finding a good priority function based on depth and model prediction. It is the key of the algorithm so it needs to be robust for it to play good moves. Also time pressure was particularly felt since I was alone doing the job of a full stack team. Finally, the fact that reinforcement learning was not covered during lectures made things harder.

Accomplishments that we're proud of

The final results are an approximated elo of about 1128 when tested against chess.com bots. This is quite a lot above my original goal of 1000 elo. Managing to deliver on time desipte doing everything by myself is also something I am proud of.

What's next for Kero chess bot

There are a lot of potential improvements that could not be made due to time pressure, so it would most likely be possible to improve it by a few hundred rating points giving it enough time.

Built With

• css
• flask
• google-cloud
• html5
• javascript
• keras
• python
• rl
• tensorflow