We liked the idea of the Oculus Rift being used for more serious applications and realized that there are few augmented reality apps for Oculus; we decided, then, to make one that could benefit people who are in a foreign country and unable to understand the population there as well as deaf people, who cannot hear what people are saying with ease.

How it works

We hooked a PS3 Eye webcam up to our computer and mounted it on the Oculus Rift. From there, we made a C# program that listens for input when you press a button. It listens for whatever language you choose in the program, parses it into text, then sends it off to be translated to whatever language is your native tongue. It is then displayed in a style similar to subtitles in a movie.

Challenges I ran into

This was my first time making a Windows 8 Store App. I had never worked with XAML before and there were some quirks with getting the webcam to work. Additionally, Richard had done very little programming with an Oculus Rift or Windows Store Applications, so this was his first major foray into development using these technologies. However, we made sure to get help from others and search diligently to find what we needed to do.

What I learned

I learned a lot about XAML, Richard learned a lot about programming in a hackathon setting, and I learned a fair amount about Speech Recognition and Translation APIs by Bing.

What's next for Convox

The next step for Convox would be to dabble into OCR and translate written foreign words into your native tongue.

Share this project: