SignDuo

Main Menu
race mode
learning mode

Inspiration

SignDuo was inspired by two things we love: Duolingo and Dance Dance Revolution (DDR).

Learning American Sign Language (ASL) fingerspelling can be challenging for beginners. Watching tutorials and memorizing handshapes is one thing, but practicing without immediate feedback is another. We wanted to make that process more interactive, engaging, and motivating.

So we created SignDuo, where your hands become the controller, and learning ASL fingerspelling becomes a friendly competition.

What it does

SignDuo is an interactive, webcam-powered ASL learning game that combines real-time hand recognition with competitive gameplay.

Instead of typing letters on a keyboard, users make ASL handshapes in front of their webcams. SignDuo recognizes supported handshapes and turns them into game inputs.

How we built it

We built SignDuo as a web application using React, JavaScript, and Vite, with computer vision powered by Google MediaPipe Hand Landmarker.

Our system follows a real-time recognition pipeline:

Webcam → MediaPipe → Hand Landmarks → Machine Learning Model → Sign Recognition → Game Logic

Challenges we ran into

We struggled with recognition accuracy, supporting two webcams simultaneously, preventing repeated detections, and integrating code from four teammates.

Accomplishments that we're proud of

We built a working game featuring 16 recognizable ASL letters, six interactive lessons, real-time feedback, and split-screen multiplayer, all within a hackathon.

What we learned

We gained hands-on experience with computer vision, machine learning, React, GitHub collaboration, and AI-assisted development. We also learned how gamification can make education more engaging.

What's next for SignDuo

We hope to support the full ASL alphabet, more signs, improved recognition, online multiplayer, and personalized lessons, while collaborating with ASL educators.