Third Eye

A device clipped onto glasses that detects and names objects pointed to by the wearer, designed specifically for young children and people with impaired vision.

Comment

Inspiration

We want to equip visually impaired people with independence, and parents with a tool to help educate their infant children.

What it does

The Third Eye module is a Raspberry Pi camera that can be clipped onto the wearer's glasses, attached to a portable Raspberry Pi unit worn around the neck like a necklace.

When the user points to an object within the camera's line of sight, the module will identify and name the object out loud through audio. Third Eye can only be accurately used on objects that are within a few meters of the wearer.

How we built it

We built the project using a Raspberry Pi camera for input and a Raspberry Pi unit for processing. We used Python, Tensorflow, and Resnet for the image classification functionality. We used Pyglet and gTTS for the text-to-speech component.

Challenges we ran into

We spent 95% of the entire hackathon downloading, installing, and setting up the necessary packages and files for our project. The process was made worse with the poor wifi at the campus building; at one point, Bryan's laptop blue screened during a 1.7GB download. A kind mentor helped us by downloading the file on his phone using data and transferring it to Bryan's laptop via a micro-USB to USB cable.

We also needed to make our Third Eye module portable, which was impossible without a portable battery. Portable batteries were not provided at the hardware booth, and we didn't have one on us either. Fortunately, the hackathon organizers were kind enough to go out to purchase one for us.

Other challenges we ran into include spending hours configuring our Raspberry Pi module, getting TensorFlow to work, and finding a functional pre-trained object classification model.

Accomplishments that we're proud of

Staying alive. For Bryan, staying awake.

On a more serious note, we are proud of building a machine learning project with raspberry pi, despite having zero prior experience with either machine learning or raspberry pi.

What we learned

“Believing is achieving.” - the mentor who helped us.

Also, bring our own hardware so we don't have to pull out our hair trying to figure out what the last person who used our Raspberry Pi did wrong.

What's next for Third Eye

Next, we will train more accurate models with larger datasets to include a wider range of objects. This will improve the functionality of the Third Eye.

We also want to implement text recognition, so that Third Eye can be used to read books out loud. This way, visually impaired people can read just like everyone else.

Built With

Share this project:

Updates