Say Chess

Featuring Automatic Voice Detection :)

Inspiration

I have been in love with the idea of playing Chess without looking at the board for a while, but being terrible at visualising it in my head, I have always struggled.

When I discovered voice agents, I figured I could finally do it! Not to mention potentially make the game more accessible for others too :)

What it does

You can play Chess by talking to an agent. You can ask for precise moves: "left knight to a3" or imprecise commands: "can you move my queen as far as away as possible?"

The AI will try to figure out whether it has enough info to play a move or if it needs to ask you for more :)

How we built it

We used bolt.new to do a lot of prototyping, with some extra manual code & LLMs here and there to fix what wasn't working. We were quite happy with the speed at which it produced our first UX prototypes :)

Challenges we ran into

Tool call is a bit more art than science. It's hard to tell how to make the AI do what's obvious. But we eventually got it to a decent state by adding lots of context to the System Prompt.

We send stuff like this to the AI: a b c d e f g h 8 ♜ ♞ ♝ ♛ ♚ ♝ ♞ ♜ 8 7 ♟ ♟ ♟ ♟ ♟ ♟ ♟ ♟ 7 6 · · · · · · · · 6 5 · · · · · · · · 5 4 · · · · ♙ · · · 4 3 · · · · · · · · 3 2 ♙ ♙ ♙ ♙ · ♙ ♙ ♙ 2 1 ♖ ♘ ♗ ♕ ♔ ♗ ♘ ♖ 1 a b c d e f g h

That's kinda cool, no?