Awards We'd Like To Be Considered For

We would like to be considered for the overall award, the Google Cloud award, the Anthem award, and the award. Submission

Our registered domain name is:


Our goal is to help children with special needs better understand and interact with the world around them.

Research has shown that children with autism are unable to learn language at the same rate as their peers. Instructional sessions with doctors can help, but they are often expensive and not widely available, especially due to the current global situation.

Behind the Name

Just like Heimdall from Norse Mythology, our product allows users to see the world in a new light.

What it does

Our solution leverages the growing widespread use of mixed reality to provide a cost-effective communication interface.

Our software uses computer vision to determine what object the user is looking at. It then displays the name of that object on the Google Glass (or other Android device).

How we built it

Our solution is a mobile application for android devices that uses Google’s Tensorflow for object-detection API. With our solution, smartphones and Google Glass can help reinforce the visual cues for everyday objects through speech and text across multiple languages.

Challenges we ran into

We ran into issues with the speech-to-text API since our program uses a dynamic input from the object detection output of the computer vision program. Our solution was to ensure that we only detect the most prominent, centred object in the frame and that the program only speaks once for every new detection.

We also ran into the issue of privacy with Head Worn Displays (HWDs) since people will be using our products not only in public spaces but also on private property. This was resolved by using Google Glass which has multiple built-in features for ensuring user privacy through encryption, providing notification through LEDs, and more.

Accomplishments that we're proud of

We're proud to be hacking for social good.

We're proud to have made a finished, deployable, and testable product.

What we learned

We learned a lot about working with various API's, and how to work as a team.

What's next for Heimdallr

We'll be reaching out to the Emory Autism Center and the Children’s Healthcare of Atlanta to test our product in the real world.

Share this project: