As our society becomes more and more globalized and interconnected, communication between everyone and anyone is crucial, especially those who share different languages or have hearing disabilities. Our app aims to combat this issue! We aim to use Hello Mundo (Mundo meaning "World" in Spanish) to advance the process of learning different languages.
What it does
The app transcribes whatever you say into a speech bubble right next to your face and instantly translates your speech into a language of your choice. The app also has the ability to read out loud the translation if you are interested in what your speech sounds like in another language in a call-and-response format. This web app can connect people who speak different languages and connect them in a natural face-to-face conversation.
How we built it
Challenges we ran into
Since we are all freshmen with no computer vision experience, we had some difficulties with creating the speech bubble and having it follow the user. We had some trouble with the webcam framerate and the constant flickering of our speech bubbles. Towards the middle, we have some issues with implementing Google's speech API with our main code that creates the webcam and the speech bubble. After this implementation, it took a while to translate the transcription because we were new to Node.js and were trying to understand the communication between the client base and the server.
Accomplishments that we're proud of
We're proud that we were able to grasp the functionality of Node.js and other other frameworks and APIs that we used after staying up all night! Seeing the text bubble float by the user's head, watching the speech get translated almost instantly into different languages, and looking at the neat, colorful designs we came up up have made this experience well worth the lost sleep!
What we learned
What's next for Hello Mundo
In the future, we aim to allow multiple people to talk to each other, each with their own bubble, and have their conversations be translated. Ultimately, we hope we can extend this app for impactful applications, such as Chrome extensions that can translate videos that you are streaming or Skype calls. This app has the capability of bridging language and communication barriers, allowing people to interact with a more diverse pool of people and culture on the Internet. We can help educate future generations through our app's simple call-and-response way to learn a new language. With some added features, it can even help people all across the world talk and learn from one another. We truly believe that our technology, even combined with something like Google Glass or HoloLens, could revolutionize communication and make each other better members of the global community.