Chess Vision

Inspiration

One of our core members Jessica Buchanan brought the idea to us after she described the mismatch between playing chess online and with a physical board. Electronic boards are available for relaying play and practicing with computer modules, but they are expensive. With a bit of AI we could provide a low-cost alternative for relaying moves and connecting a physical board to computer training programs. With that in mind, Chess Vision was born!

What it does

Chess Vision allows users to send data about a chess match to a computer with the only requirement being a camera and a board. With an image classifier, Chess Vision identifies the state of the board and updates piece position, ultimately making for a comparable experience to that of a DGT chessboard at a fraction of the cost!

How we built it

Built with Keras Sequential API - layering convolutional 2D neural network layers of 32-28 neurons. Hidden layers have ReLu activation and the output layer has softmax activation function. Input layer accepts 60x60 pixel RGB images. Webcam feed of the chessboard is divided into 8x8 grid of chess squares. The individual squares are resized to the 60x60 input size and submitted to the CNN which outputs a prediction for the three classes: empty chess space, chess space with a black piece, chess space with a white piece. Using changes in status of board spaces from empty --> occupied or occupied --> empty, can determine which piece was moved and where it moved to.

Challenges we ran into

Maintaining accuracy after initial moves, the image classifier is accurate at confirming that the starting chess squares are occupied by the right color pieces (over 90%) but accuracy decreases after the first move. There are challenges with dealing with low-contrast between piece and board in the image which leads to incorrect classification of occupied spaces as empty. Refinement needed for determining moves made.

Accomplishments that we're proud of

Our final demo had much greater accuracy than when we started, especially in the initial setup test and the first set of moves. This was our first image classifier and we learned a lot about data preprocessing of images and working with live video feeds.

What we learned

Many members of the team had never worked with OpenCV, TensorFlow or Keras.

What's next for Chess Vision

In the short term, we’re planning on adding a stat tracker, to relay that information to a website for play or broadcast. We’re also planning on adding audio and visual cues to let users know when they’ve made a wrong move. We would like the user to be able to load a PGN (portable game notation) file and have the program assess accuracy in learning an opening variation or endgame sequence against the PGN. As we develop this application in the far future, we hope it will be able to use that data it collects to give users turn-by-turn training of chess, akin to the training you can find on chess.com. In addition, functionality such as playing with friends and running chess drills are all contenders for future updates.

Built with python , cv2, opencv, Keras Sequential API, Tensorflow