Accessaify

Our AI technology transforms digital content into more accessible formats, creating a more inclusive online experience for everyone.

Comment

Introduction

Over 5% of the world's population, 430 million people, are part of the Deaf and Hard-of-Hearing (DHH) community. By 2050, this number is expected to surpass 700 million, meaning 1 in every 10 people will have disabling hearing loss. Despite advancements in technology, the DHH community still faces significant communication barriers due to the lack of accessible and seamless ASL (American Sign Language) translation tools. Existing solutions often require expensive hardware, lack real-time efficiency, or fail to capture the nuances of sign language—making them impractical for everyday use. We wanted to change that. Our inspiration stemmed from the desire to create an AI-powered ASL chatbot that enables users to communicate in sign language and receive responses in ASL, bridging the gap between ASL users and AI-powered systems like ChatGPT or Claude. With Accessaify, we aim to make AI-driven conversations truly inclusive and accessible to all.

What it does

Accessaify is an AI-powered sign language chatbot that enables real-time, two-way communication between ASL users and ChatGPT. Here’s how it works: Sign Language Recognition – The user signs in front of their webcam, and our trained machine learning model detects and translates their hand gestures into text. ChatGPT Integration – The recognized text is sent to OpenAI's ChatGPT API, where it processes the input and generates a response. ASL Response Generation – The chatbot's response is then converted back into ASL using a 3D hand animation system or visual gestures, allowing the user to "see" ChatGPT’s answer in sign language. This cycle enables conversations between ASL users and our AI powered chatbot

How it works

We combined computer vision, deep learning, and natural language processing to make Accessaify a reality: MediaPipe & OpenCV: Used to capture and preprocess real-time video input for hand tracking. Machine Learning Model: Trained on ASL dataset using Random Forest, SVM, KNN, and Neural Networks, improving gesture classification accuracy. ChatGPT API: Integrated OpenAI’s API to generate natural language responses. ASL Response Animation: We explored 3D hand gesture animations and pre-recorded ASL video snippets for seamless response visualization.

Github: (Backend): https://github.com/parshG/SacHacks
(Frontend): https://github.com/AfreenInnovates/accessify

Built With

Share this project:

Updates