During the AutoTrader Workshop, we were amazed by how much technology exists for the visually impaired, and we wanted to expand and build on that, to hopefully improve their lives and give them an aid to visualize their surroundings.
What it does
Lens has two functions. The first function is an echolocator that sends out an ultrasonic wave, to measure the distance they are away from objects in their path, and reads it back accurately to them. The second one uses a DSLR and Amazon recognition to let them take a 'snapshot' or their surroundings, that will then be read off to them, in order for them to visualize their area. This is all read off to them from a speaker.
How we built it
Supersonic sensors connected to Arduino running C++ Code Connected to Windows 10 Laptop, running VirtualBox (Ubuntu) communicating over Serial Python script reading the Serial, and responding to web requests from remote SSH Tunnel to nexus.wdg.ovh to allow access from the internet "Remote" running C++ Code and communicating with nexus.wdg.ovh and back through the tunnel Bash scripts executed by their Python overlord which call the aws-cli to interact with AWS, Rekognition, Polly and the User. Camera connected to Windows, being passed over Web to the VM and being saved by wget.
Challenges we ran into
We ran into an issue when trying to use a raspberry pi, when there was an issue with connecting to wifi, in order to poll data from the sensors.
Accomplishments that we're proud of
We are very proud of the fact that we managed to create a fairly accurate ultrasound wand using only basic Arduino components. We are also very impressed in our Lens Snapshot, which has proved to give a fairly accurate idea of the area the camera is in.
What we learned
That CV is powerful and great, as is Amazon Web Services, and that Visually Impaired people are incredible in how they still manage complex day to day tasks without the power of vision.
What's next for Lens
For Lens, we would ideally like to implement a better speaker, to allow the audio to be clearer, maybe implementing a feature to allow headphones to be plugged in. We would also make the camera smaller, maybe using a GoPro so that it can be easily mounted. We would also create a better case for Lens, and an easier way to transport it.