Nearly 40 million people in India alone are visually impaired(285 million worldwide). It is hard for them to do their own jobs. They find themselves dependent on someone almost always. Devices that help the visually impaired people by scanning the environment and guiding them accordingly do exist but they are priced at several thousand dollars. The cost factor makes accessibility a question mark.
This inspired us to develop VC4U (We See For You) - an affordable device for the Visually Impaired
What it does
VC4U has 2 components - An APP and a Spectacle The spectacle is embedded with a camera and a touch sensor. Whenever the user wishes to use the device he touches the sensor. This action triggers the app to listen to the voice command the visually impaired person is about to give. With NLP the speech data is processed and is converted to commands. Here are a few commands (and their motive): -> Detect objects: Gets image from the onboard camera in the spectacle and detects objects, which is then converted as Voice feedback. ( Sign Boards, Person, Traffic Signs, Trucks, Bus are few classes to mention). Tflite (Tensorflow lite) and YOLO are used to achieve the task. ->Where am I? : Gives the exact location of the user via voice commands
How we built it
For Capturing image ESP-CAM32 was used. The captured image is sent via HTTP to its clients. For NLP wit.ai was employed. The App(Flutter based) uses YOLO - custom trained on Open Image dataset and was integrated with the app using tflite.
Challenges I ran into
Getting hands-on the required Hardware and completing within the stipulated time were the biggest challenges.
Accomplishments that I'm proud of
The main objective was to provide a low-cost solution, which was possible - the cost price was under 1000 INR.(14$)
What I learned
Wit.ai is a great platform for any solution that involves NLP and was really user-friendly, it was a good experience exploring it. Flutter was a totally new experience for me and my team. We had great fun working with flutter. And worth mentioning - ESP's capabilities and YOLO surprised us.
What's next for VC4U - An aid for the Blind
Next, we are looking into developing improvised versions and making it available for the visually challenged community in an attempt to make the world a better place.