The capabilities of the Bose Frames were the inspiration for this project. We wanted to challenges ourselves by making an immersive and engaging experience purely through the use of audio. The IMU data coupled with the high quality audio provide by the Bose Frame made them the ideal product for this type of project.
What it does
This app emulates the experience of being locked in a dark dungeon. The user stumbles across a pair of glasses that end up belonging to an unknown soul. The glasses' owner guides the player through escaping the dungeon using only auditory commands. The user must move and twist their body and head to locate sounds identifying the key and the locked door to exit the dungeon. Once the player unlocks the dungeon's door they may be surprised at what comes next.
How I built it
This app was built in Android Studio and programmed in both Java and Kotlin. The Bose AR SDK was used to get the IMU data from the Bose Frames. The Google VR Audio (gvrAudio) API was used to create spatial sounds. Once these systems were established. The IMU data was filtered, gravity-compensated, and integrated to calculate the position and orientation of the user. The user's position and orientation is then used to create the spatial audio effect using gvrAudio. Once these systems were functioning properly narration tracks were recorded to guide the player through the game.
Challenges I ran into
One of the largest challenges we ran into was lack of experience developing apps in Android Studio. We overcame this hurdle by using our preexisting programming skills and consulting as much documentation as we could. Additionally, deriving the user's position from the IMU data was a large hurdle. This challenge was overcome by experimenting with various filters as well as adding gravity compensation to the accelerometer data.
Accomplishments that I'm proud of
We are proud of integrating spatial audio with the IMU data given by the Bose Frames in order to create an immersive experience as well as overcoming our challenges in calculating the user's position from the IMU data.
What I learned
Going into this project both team members had little to no experience developing android apps. This project allowed us to learn the basics of android development.
What's next for The Ghost in the Glasses
If we are able to get our hands on more Bose AR hardware we would love to expand on our current Bose AR application as well as develop ideas for new Bose AR applications. One application we would have interest in pursuing is an application design to help blind people navigate. This could including guiding them to the nearest bathrooms and exits as well as notifying them of there surroundings.