White Cane

White Cane is an adaptive augmented reality messaging app that notifies users when they are about to run into a wall.

Comment

Inspiration

Everyone has done it or has at least almost done it. The "it" I am referring to is the all too embarrassing experience of running into a door while engrossed in a texting conversation. Our app is built to solve that problem using cutting edge technology integrated with a clean and easy to use design. A white cane is a traditional name for a cane that a blind person uses, in the same way, our app is a white cane for a texter who is temporarily blinded due to lapse in attention.

What it does

The basis of the app is a basic texting application built on Firebase. The chat data is stored in a Firebase database. The augmented reality is where the magic truly comes in. The app provides a view of what the camera sees in front of it, on top of that view the messages are overlayed. Due to the possible changes in the color of the background, the text color is automatically adjusted based on the lighting conditions of the room. The app automatically detects and labels the surfaces that it finds in its surroundings. If that surface is detected to be a hazard to the user, the user is notified(via vibration) and hopefully saved from harm!

How we built it

The whole app is written in java in android studio. We built the chat part of the app with Firebase, and all of the augmented reality code was written using ARCore, and sceneform.

Challenges we ran into

We ran into challenges with the detection of vertical planes that could be hazards. Not being familiar with ARCore, we weren't sure how to implement some of the plane finding features without using button press functions. After several hours of research, documentation study, trial and error, and frustration we were finally able to find vertical planes independently of button presses. One problem we are currently working on addressing is making sure that the hazard notification ends immediately after the hazard is no longer present.

Accomplishments that we're proud of

We utilized ARCore to detect the light intensity of the frame. Based on this intensity we created a quadratic function that mapped intensity values to a value in between black and white for the text. Originally, the function was linear but it was changed in order to prevent the case when both the intensity and the color were of a 'medium' value. This function perfectly matched what we thought was effective for the most readable text at each light intensity level.

What we learned

Starting off as android beginners, we learned a lot from this project. We learned about how an android app works and how to use technologies such as firebase and ARCore to connect the app to the internet and the world around it respectively. We learned about how we can use ARCore to make predictions about the world around us in order to make simple tasks even simpler.

What's next for White Cane

We have worked long and hard for White Cane to become what is today. However, there are definitely areas for potential. For example, we could make this app usable with SMS. In other words, the app would be able to send and receive texts, so the user could take advantage of White Cane as a client-side modification. Additionally, we would love to develop an iOS version of White Cane that could run as an iMessage app.

Share this project:

Updates