Our inspiration was to have computers interact with the real world. We wanted to bridge the gap between the amazing computational power that computers offer in terms of chess analysis and the physical experience of playing on a wooden chess board.
What it does
The raw video stream is first processed in order to detect the corners of the chess board. Then, those corners are mapped to a perfect 320x320 square representing the chess board. The occupancy state of the board is then determined at each frame, which is used to find out which move has been made using state-machine logic. Finally, the position is sent to a server we built which analyzes it and finds a "best" move. This move is then communicated to the user.
How we built it
Used several image/video processing techniques from the opencv library along with raw matrix transformations to obtain a state in which each square on the chess board is being tracked. The differences in occupancy states of the board are used to find which moves were made. These moves are then sent to back-end server to be processed.
Challenges we ran into
There was a lot of noise that had to be cleaned up when trying to process a video stream and detect changes. Also, finding the thresholds to determine if there is a piece on the square or not was a tedious process (it is difficult to detect white pieces on white squares and black pieces on black squares).
Accomplishments that we're proud of
Actually having a computer track a chess game through a camera is pretty cool. Also, hooking up the most powerful chess engine in the world to your back-end to find moves is nice accomplishment we're proud of.
What we learned
Image/video processing is difficulty ... very difficult.
What's next for Ultimate Chess Hack
Having more subtle ways for the back-end to communicate the chess move to the player for some not-so-fair game-play ;)