Cerebral palsy and multiple sclerosis are both conditions that can impair a person's fine motor skills. These conditions can keep people from participating in everyday activities. One of these activities includes chess, a simple but beautiful game that can be a hobby for any person. However, some people cannot play games such as this due to disabilities, namely those with cerebral palsy and other such diseases. After seeing many people closed off from playing games that require fine motor skills, I decided to make an application that utilizes a person's voice instead.

What it does

VChess uses the power of your voice to move pieces on a virtual chessboard so that you can play chess with your friends without feeling left out.

How I built it

I trained a deep learning voice detection model using thousands of hours of chess notation audio with a 97.3% accuracy rate when tested with the training data set. I then created a web interface using tailwind and a custom javascript chessboard linked to my model.

Challenges I ran into

The most difficult problem I had to face was incorporating the model into the web application. To achieve this, I had to make a custom chessboard in JavaScript so that I could have fine control over each aspect of the game. This way, users can control any aspect of the game they want through their voice.

What I learned

I learned a lot about how deep learning pipelines worked in Python through this project as well as web development and front-end design.

Share this project: