[Gambit]

A close-up of Bracket Bot’s gripper positioned over the chessboard in simulation.
Gambit at the board: Bracket Bot sees, plans, and plays chess in a simulated physical environment.
Gambit tracks the physical board, validates each move, and shows Stockfish’s response alongside live camera views.

Inspiration

Chess engines have been stronger than humans for decades, but playing one still means staring at a screen. We wanted to build an opponent that could sit across from you, watch the board, and move its own pieces.

What it does

Gambit plays chess against a human on a physical board. It observes each turn with Bracket Bot’s camera, identifies the move, checks that it is legal, and asks Stockfish for a response. The robot then announces the move and plays it using its arm and gripper.

How we built it

We split the project into three parts: vision, chess logic, and robot control. The vision system compares the board before and after each turn. Legal-move filtering resolves those changes into a chess move, Stockfish selects a reply, and the robot converts the source and destination squares into pick-and-place motions.

Challenges we ran into

The hardest part was not playing chess—it was handling the physical board. Camera perspective, shadows, crowded squares, grasp alignment, and small placement errors can all desynchronize the real board from the software. We addressed this by constraining the board setup, validating every detected move, and checking the board again after the robot acts.

Accomplishments that we're proud of

We connected perception, game logic, and manipulation into one loop. A player can make a move without entering it manually, and Gambit can understand it, choose a response, and physically play back.

What we learned

Robotics requires every layer to agree about the state of the world. A strong chess engine means little if perception is uncertain or a piece lands on the wrong square. Verification and recovery ended up being just as important as the main behavior.

What's next for Gambit

Next, we want to make Gambit less dependent on a controlled setup, improve grasp recovery, and support more boards and pieces. We would also like to add adjustable playing styles and a coaching mode that explains positions while playing.