Eye sight or vision is one of the most basic human senses because it allows us to connect with our surroundings, keep us safe, and help maintain the sharpness of our minds. This gift, unfortunately is not possible for the blind. So, we made an app for them that helps in identifying objects around them.
What it does
Blind-Aid is a web application that is used to convert objects that are detected from the camera to speech which enables blind and partially blind people to know about their surroundings. The application automatically detects the objects which is detected using the Object Detection API and finally these responses are converted to speech by means of the Google text to speech API.
How we built it
We used the following techstacks:
- TensorFlow TensorFlow is a free and open-source software library for machine learning. Here we used it for develop and train ML models.
- OpenCV OpenCV was used for image processing.
- gTTS We used the Google text-to-speech Python module to convert the filtered text and confidence level we got from processing the models, saved the audio
- Flask Our complete backend was made with the Python web framework Flask, and we used it to integrate the ML models to the web application.
Challenges we ran into
- Integrating the front end with the backend was a bit tricky.
- Setting up the gTTS API to convert the received responses to audio file.
Accomplishments that we're proud of
- We were proud to find out the trained models actually worked with real life webcam feed and succeeded in detecting the objects.
- That the entire project worked out fine.
What we learned
- Learned how to train models and also how to use tensorflow.
- We were able to expand our knowledge in Flask.
- Learned about gTTs ,converting base64 strings to png etc.
What's next for Blind-Aid
- The existing model could be refactored into an mobile application, making it possible to reach more users.