In todays advanced world, the need of independent living is recognized in case of visually impaired students who are facing main problem of social restrictiveness. Due to lack of necessary information in the surrounding environment visually impaired people face problems and are at disadvantage since visual information is what they lack the most. With the help of my project, the visually impaired can be supported. The idea is implemented on voice assistant which is capable to assist using voice command to learn Grammar topics like Parts of Speech, Name Entities, Classification and many more. It may be the effective way blind people will learn new things.
What it does
It takes Voice Input from User and Asks him which Operation to apply on his spoken text ,that data is analyzed using Expert.AI's Natural Language Processing API , which is platform to add language intelligence to our applications. The Process works like this:
- Initially the User is asked for Input Sentence using Speech
- The Speech to text Algorithm converts it into text.
- Then the user is asked to choose between Grammar Operations like Parts of Speech, Name Entities,etc.
- The Input is converted into Text and Checked using Similarity Algorithm if it is valid operation.
- That Operation is made active and API is called using Input Text taken initially
- The Output of API is formatted and Converted to Speech.
- Then Finally user is asked to Choose to Continue, Start over or Exit and this process continues.
How we built it
This Block Diagram of Project
The API is called using Expert.ai 's Python Client. I followed the Authentication steps Documentation and it worked well. The instructions are well maintained by expert.ai community.
Text To Speech:
For this I have used Python's package called pyttsx3 .It works according to code flow and user input.
Speech to Text:
I have used the python packageSpeech_Recognition ,and connected it with my laptop's Microphone. It works very well than any other STT API and is much fast.
As User's Speech input may not match the required operations perfectly so I have used a Text Similarity Algorithm named add code
Challenges we ran into
- API calling and Authentication Issues as Documentation on Website and GitHub were little Different
- Connecting Audio with Python was difficult due to multiple Linux dependencies
- API Calling Limit Error ,had to change IP Address frequently.
Accomplishments that we're proud of
- First Time implemented Audio Processing
What we learned
- API Calling of Expert.ai
- Python Audio Processing TTS and STT
What's next for GrammaXpert Chatbot
- Easy GUI Mobile App
- Porting on Embedded Devices like Raspberry Pi
- Adding Dynamic Visual Animations
Log in or sign up for Devpost to join the conversation.