Team Charizards - ASL Google Search Extension 🔍

Track: Best Hack for Social Good - Signify

Project Idea

What if you could talk to Google with just your hands?
No typing, no clicking, no yelling "OK Google!" across the room.
We built a Google Chrome Extension that recognizes American Sign Language (ASL) using your webcam and turns it into automatic Google searches.

It’s like charades — but smarter, and with results.

Team Charizards

Chetan – Probably taught MediaPipe how to meditate
Gourav – Whisperer of WebSockets
Mayuk – Palette and Meme Machine
Charu – ASL Ninja and Debugging Queen

Objective

To help differently-abled individuals surf the web without typing or pressing Enter.

Because inclusion isn't just cool — it's necessary.

How It Works

Tech Stack

Layer	Tools Used
Frontend	JavaScript, HTML, CSS
Backend	Python, Flask
AI Model	MediaPipe (Hand Detection), KNN
Communication	WebSocket
Data	Custom ASL dataset

Methodology

Preprocess custom ASL dataset (because good data = happy AI)
Train the model using MediaPipe for hand landmark detection
Use K-Nearest Neighbors (KNN) for recognizing signs
Build a Chrome extension that captures your hand signs via webcam
Detected words are automatically searched on Google

Yup, it’s basically giving your browser sign language superpowers.

MVP (Minimum Viable Product)

Single-hand ASL gesture recognition
Real-time webcam capture
Instant Google search triggered by signs
Works inside Chrome with no extra downloads

Future Plans

We're just getting started. Coming soon:

Double-hand gesture recognition
Training with full ASL dataset
Turn search results into audio output (for blind + mute users)

Accessibility just got an upgrade.

Big Questions?

What if signs are ambiguous?
→ KNN + smoothing filters to the rescue!
Can this go mobile?
→ Definitely. We’ve got our sights on that too.
Is it fast enough?
→ Like a cheetah on Red Bull (yes, it's real-time).

Sneak Peek (Add Screenshots/GIFs Here)

https://drive.google.com/drive/folders/1rHCF73eaK_ackR0hBuWss9xrKvpLRhk_?usp=sharing

Why It Matters

Typing is easy for many of us. But for those who communicate through ASL, web search can be a hurdle.
With this extension, we’re making the internet more inclusive, fun, and just a bit magical.

Because the web should be for everyone — including the hands that speak.