youtube thumbnail

Inspiration

Hand Town grew from a simple idea: a family’s living-room TV could respond when someone makes a handshape. Instead of making fingerspelling feel like a worksheet, the app turns each choice into a small moment in a garden: a bird arrives, finds a nest, sees the sun, and flies home. The TV remote is always an option, so the experience doesn’t depend on having a phone connected.

What I built

Hand Town is a family-friendly fingerspelling game. Its TV app displays an animated garden, eight illustrated handshapes, and a short B → C → D → Y story. A D-pad can control the experience on its own.

The prototype also includes a phone page with local hand-landmark detection and a shared demo classifier. A small Node.js relay handles pairing and sends validated sign events to the TV. Camera video and landmarks stay on the phone; only small JSON events are sent through the relay.

What I learned

Building for a TV means designing around distance and a remote: focus must be obvious, text must be readable across a room, and every action needs a clear D-pad path. I also learned that recognizing a handshape in a sample set is not the same as reliable recognition for everyone.

Challenges and current limits

The shared classifier uses samples from one adult. It has not been validated across different people, lighting conditions, or phone cameras, so recognition may vary. Hand Town is a prototype fingerspelling game, not an ASL fluency course, and its handshape cues have not yet been reviewed by a Deaf signer or ASL educator.