We believe in all products we build and hope to solve real world problems as much as possible. We noticed that there are times when people from different countries, speaking different languages, find it difficult to share their views due to language barriers. This can lead to things becoming a bit awkward between them. Being hackers and using the power given to us by coding, we came up with an innovative solution to this problem - OcWord: is an augmented reality system which uses oculus rift to enhance communication amongst people. Another motivation was that OcWord could be used to help deaf people understand what others are speaking by reading texts appearing in augmented reality.
What it does
The way it works is that there is a web camera attached on top of the oculus which captures real life video and then renders it to the oculus. This makes the person wearing the oculus get the presence of being in his real surroundings. Whenever someone comes in front of the person wearing OcWord and speaks something, his/her speech is converted to text using google's speech to text API and through an option can be translated to a preferred language. These features allow the people to communicate unrestricted.
How we built it
We hacked on an oculus to try out the possibilities of the virtual reality. To make it more real life like, we added a web camera to device which streams live video to the user seeing through the oculus. We then researched about and used the google's speech to text API and google translate API for converting speech to text and in any preferred language. Then we programmed in unity to map the text on the video creating the affect of augmented reality.
Challenges we ran into
There were a couple of challenges we ran into through the development cycle - choosing an interface for development, making sure that speech to text works smoothly, designing the scene in Unity, ensuring the distortions are minimized and the user sees a clear real life-like picture were hard interesting challenges which forced us to approach the problem from a different perspective.
Accomplishments that we're proud of
This was our first attempt to hack on the oculus, we are really proud of what we could achieve within these 36 hours. Creating the scene in Unity and getting it to appear on the oculus, and best of them all was to get the speech to text translation and that also in multiple languages appear smoothly on the screen.