• 

  Landing Page

• 

  Initial Home Page

• 

  Streak set up

Lexa - Interactive ASL Alphabet Learning

Inspiration

Learning American Sign Language (ASL) is crucial for communication accessibility, yet traditional learning methods often lack the interactive feedback needed for proper hand positioning and gesture accuracy. We created Lexa to make ASL learning more accessible and effective through real-time computer vision technology.

What it does

Lexa is an interactive web application that teaches users the ASL alphabet through real-time hand tracking and gesture recognition. Users open the app in their browser, allow camera access, and start practicing ASL letters with instant feedback on whether their signs are correct. The app guides users progressively through the entire ASL alphabet while tracking their learning progress and identifying areas for improvement.

How we built it

We built Lexa using React and Vite for the frontend, with Tailwind CSS for styling. The core functionality relies on MediaPipe Hands for 3D hand landmark detection via webcam. We implemented a custom K-Nearest Neighbors (KNN) classifier in JavaScript that processes hand landmarks into normalized features and compares live gestures against pre-collected training data. A Node.js/Express backend serves the training data, while the entire ML pipeline runs client-side for privacy and real-time performance.

Challenges we ran into

Getting consistent hand detection across different lighting conditions and camera qualities required extensive parameter tuning and robust preprocessing. Building a classifier that could distinguish between similar ASL letters while being tolerant of natural hand variation was particularly challenging. We also had to optimize our ML pipeline to maintain 60fps performance while running complex computations entirely in the browser.

Accomplishments that we're proud of

We successfully built a complete gesture recognition system that runs entirely client-side, ensuring user privacy while delivering high-accuracy ASL letter recognition. The app achieves real-time ML inference at 60fps in web browsers and provides a seamless, engaging learning experience that genuinely helps people learn an important communication skill.

Built with: React, JavaScript, MediaPipe, Tailwind CSS, Node.js, Express, KNN Classification

Try it out: [https://github.com/seb1124/Lexa]

Built With

• daisy
• express.js
• javascript
• knn
• mediapipe
• react
• tailwindcss
• vite