Go Kitty Go! An ASL Fingerspelling Game

• 

  Simon signing F

• 

  Simon signing R

• 

  Losing is hard

• 

  Original design concept

American Sign Language is difficult to learn, but through gamification, we can ease the process. Beginners in ASL start by signing the alphabet, which can be used to spell out words in the process of ‘fingerspelling’. In ASL, fingerspelling is used both for words that may not have signs, like names, as well as communicating words in a pinch when signs are forgotten or not known. When all else fails, fingerspelling is there, so learning to fingerspell quickly is important.

Go Kitty Go makes fingerspelling practice fun while incorporating Deaf history. Taking inspiration from Google’s Dinosaur Game, we’ve created a game centered around deaf stuntwoman Kitty O’Neil’s quest for the land speed record. Unfortunately, pesky word obstacles are blocking her way and the only way Kitty can pass is if the player accurately fingerspells the word. For this evaluation, we’ve trained an AI judge using Google’s MediaPipe library and an in-house ASL alphabet dataset.

Our game represents only the first step in how computer vision can enhance ASL learning, which is so much more than fingerspelling. In future iterations, we hope to improve our game by recognizing signs for words and even sentences.

Domain.com name: https://gokittygo.tech

Hosted on Google Cloud CDN

Built With

• google-cloud
• javascript
• mediapipe