hints
display
asl letters
asl words

CyberSign Protocol

Inspiration

We wanted to make learning accessibility tools feel cool, not clinical. Inspired by 90s hacker movies where typing fast saved the world, we asked: What if we could bring that same high-stakes energy to learning ASL? We wanted to turn the quiet act of signing into a loud, visual, interactive experience.

What it does

CyberSign Protocol is a browser-based game that teaches the ASL alphabet.

The Mission

You are breaking into a secure mainframe. The 'firewalls' are letters. To bypass them, you must mimic the hand signs shown on screen.

The Modes

LETTERS Protocol: Use precise hand shapes to crack alphabet firewalls (A-E).
WORDS Protocol: Engage full-body tracking to execute command phrases (HELLO, YES).

The Tech

Using computer vision, it tracks your finger joints in real-time. It doesn't just "see" your hand; it calculates the geometry of your fingers to distinguish between an 'A' (fist) and an 'E' (claw).

Vector Geometry

We didn't use a black-box AI model. We wrote custom math to calculate the angles and distances between landmarks.

For example, to detect the letter 'C', we calculate the distance between the thumb tip $(x_1, y_1)$ and index tip $(x_2, y_2)$ using the Euclidean distance formula:

$$ d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2} $$

We then verify that the distance $d$ falls within a specific threshold ($0.1 < d < 0.25$) while ensuring other fingers are partially extended.

The Feedback

It rewards you with satisfying, crunchy 8-bit audio and visual glitches, turning rote memorization into a dopamine-fueled game loop.

How we built it

MediaPipe Hands: For the skeletal tracking.
Vector Geometry: Custom math to calculate angles between knuckles and fingertips.
Procedural Canvas: The "Matrix Rain" and "CRT Scanlines" are drawn pixel-by-pixel in JavaScript, ensuring it runs smoothly on any device.

Challenges

The hardest part was the "A vs. S" problem. In ASL, 'A' and 'S' look incredibly similar (both are fists, but the thumb position changes). Writing the geometric logic to detect that subtle thumb difference took hours of tweaking and testing with different hand sizes.

What's next

✨ Two-Handed Encryption: Adding complex signs that require both hands to unlock "Boss Level" firewalls.
📝 Custom Dictionaries: Allowing users to upload their own word lists (e.g., spelling their name) to practice specific signs.
🔗 Sentence Assembler: A mode where users must chain together words to form complete sentences.