AR Assisted Chess

• 

  A snapshot of our code

• 

  the setup to record the board

• 

  The computer's interpretation of the board

Inspiration

We're all avid chess fans but we're not all the best - its not common to look at chess engines to look back at your game and improve. However, that isn't convenient and means that few people take that time to improve. To further spread the game we love to people of all skill levels, we decided to create a easy to use system that can tell you what moves you should do next.

What it does

It is an AR-based system that when above a chessboard, it will tell you the next optimal move in real-time and overlay a helpful arrow onto the image of the chessboard.

How we built it

We trained a deep neural network using GoogleColab to be able to look at a board and identify the pieces, using keras and openCV, a computer vision module in python. We used this together with python's chess module to identify optimal moves and then used openCV once again to overlay on top of the live video feed.

Challenges we ran into

We had no idea how to conquer chess board recognition at first. We tried taking horizontal and diagonal perspectives, but parallax always was a wall. To climb over it, we trained a large data set from a top-down angle, using a DIY camera rig. We also had difficulties dealing with various lighting conditions. We had to incorporate flashlights into our rig to overcome this.

Accomplishments that we're proud of

We are proud that we managed to successfully impleement computer vision algorithms that none of us were entirely familiar with before this hackathon. We're also proud of the simple fact that this project might help people who love this game as much as us but not be good at the game.

What we learned

We learned that shouldn't aim too high sometimes, and sometimes, simple solutions can help overcome complicated issues - after all our camera rig was probably the simplest makeshift solution to complicated computer vision problems that still stump professionals.

What's next for AR Assisted Chess

We would like to be able to improve upon our computer vision techniques even further, o we can move from having only top-down views to being able to view board's from any angle, making it even easier to use. Ideally, this would be in the form of a iOs or Android application.

Built With

• googlecolab
• keras
• opencv
• python
• python-chess