Theia

A Google Glass app that helps visually impaired people know what is around them

Comment 2

Theia (noun): Titaness of Sight.

Inspiration

Visually impaired people do not have the same opportunities as other people to experience the beauty and variety of their surroundings. We wanted to build an app that would bridge this gap and give them a chance to visualize the things around them.

What it does

Theia is a Google Glass app that helps visually impaired people see what is in their environment. It captures a live stream of whatever the user is looking at, and it automatically dictates what objects are in the environment using natural language.

How we built it

We built the app in Android Studio, coding in Java. We're sending images from Glass to the Clarifai API, where it performs an image recognition search and returns words describing the image. Then we structure the words into a sentence and dictate it using Android's Text To Speech.

Challenges we ran into

We had trouble setting up the camera and making Clarifai API requests, which we were eventually able to overcome through the use of ingenuity, documentation, and Stack Overflow.

Accomplishments that we're proud of

We're proud of creating an app that recognizes images with decent accuracy. We're proud of creating something that will be very useful to the visually impaired.

What we learned

We learned how to develop for Glass applications, how to use the Clarifai API to recognize images and get the results, and how to set up a video stream using the Android API.

What's next for Theia

We're planning on developing Theia for mobile devices and moving beyond the discontinued Google Glass hardware into building our own proprietary hardware in order to optimize the experience. We would also like to make it more accurate by training our own neural network with large data sets so it can generate better captions.

Share this project:

Updates