• 

  x representation (Alphabets)

• 

  Home representation (words)

• 

  c representation (Alphabets)

• 

  one representation (Numbers)

• 

  Time representation (words)

• 

  speech and text conversation

Inspiration

The inspiration behind SignKit was to bridge the communication gap between hearing-impaired individuals and the general public. We wanted to create an accessible and intelligent system that makes sign language learning and communication easier using modern AI and 3D technologies.

What it does

SignKit converts speech, text, or videos into Indian Sign Language (ISL) using an AI Glossing Model and displays the output through a 3D animated avatar. The system processes user input from any of these sources, generates ISL gloss, and renders real-time sign language gestures to enhance accessibility, communication, and learning.

How we built it

We built the system by integrating Speech Recognition for voice-to-text conversion, Natural Language Processing (NLP) for text analysis, an AI Glossing Model for converting English text into ISL gloss, and a 3D Avatar Engine to animate sign gestures. Video input processing is included to extract content from live or recorded videos. A gesture database maps gloss words to corresponding animations, and an administrator module manages system updates.

Challenges we ran into

One of the major challenges was accurately converting English grammar into ISL gloss structure. Synchronizing gloss output with smooth 3D avatar animations was also technically complex. Ensuring real-time performance and maintaining gesture accuracy required careful system design.

Accomplishments that we're proud of

We successfully developed a working prototype that converts speech, text, or video into ISL gloss and displays it through a 3D avatar in real time. Integrating AI-based gloss conversion with 3D animation within a single system is one of our key accomplishments, demonstrating seamless, interactive sign language communication for enhanced accessibility and learning.

What we learned

Through this project, we gained practical knowledge in Artificial Intelligence, NLP, Speech Recognition, 3D animation integration, and system design. We also learned the importance of accessibility-focused technology development.

What's next for 3D Avatar-Based Sign Language Learning and Conversion System

In the future, we plan to improve gloss accuracy using advanced deep learning models, support multiple sign languages, enhance avatar realism, add mobile application support, and integrate real-time sign-to-text conversion for two-way communication.

Built With

• 3d
• animation
• api
• application
• avatar
• based
• browser
• character
• css3
• es6
• glb
• gltf
• graphics
• html5
• interface
• javascript
• language
• logic
• models
• platform
• rigged
• sign
• signkit
• speech
• text
• three.js
• to
• user
• webgl