Voice is an app that helps people with disablity to speak with recognizing thier gestures giving out text and speech

  • Some of the challenges faced by people with disability to speak:
    1. A lack of communication medium leads to confusion
    2. They cannot use the call feature.
    3. When using ASL, interpreters are required. They are scarce even after the implementation of ADA and quite expensive.
    4. HealthCare, court rooms pose a much different and much deadlier challenge.
    5. Hospitals, schools, and healthcare organizations need to include cultural competency and humility training with authentic community representation.


  • Easy to use ,just use ASL gestures to convert it to text and speech


Data was created by capturing about 1200 frames for each action CNN was used to train the data & Keras library was used Each of the action was mapped to the corresponding text A live stream of video is captured using OpenCV and its interpreted frame by frame the gesture is recognized and text is given as output This text is sent back to AZURE text to speech API for conversion of text to Speech.

Techstack used

-Python -OpenCV -Keras -Javascript -Flask -HTML,CSS,Materialize -Git,Github -Azure Cloud services(Text to speech API) -DocuSign API for Terms & Condition related to the use os images of user.


Screenshot 1 Screenshot 2 Screenshot 3 Screenshot 4 Screenshot 5

Future Work

-converting the application into a fully functional web application(end to end)


Abhay R Patel Uddhav Navneeth Aditya Srivastava

Built With

Share this project: