Communicate.AI: A better way to talk
With the recent onset of augmented reality headsets, which are intended to give users seamless integration between technology and the real world, we believed that we could use this technology to solve the language barrier between many different cultures worldwide.
Communicate.AI is a program that can detect and translate language live, by either providing a written subtitle below a person's face or by using text to speech in 133 different languages.
It is built off of a number of APIs, including Google Cloud's text-to-speech and speech-to-text, Google Translate's API, and OpenCV's API for facial detection. In addition, we optimized performance of the program by using multi-threading while connecting all of these APIs together.
Some challenges that we ran into was figuring out how to share data between threads/processes when using multithreading/parallel processing, as once a threaded function starts you cannot send more data or arguments to it. We looked for solutions, then realized we can use a queue reference to share it throughout the threads that need to communicate with each other. The next challenge that we ran into was making the camera function with multithreading. This was alleviated through adding the video stream as a separate thing that is initialized outside of a thread, but then sending a stream of frames from the video stream to a thread for processing. Finally, we had some problems working with the complex authentication that the Google APIs require. Luckily, we were able to perservere
Thought this project we learned to better integrate functionality that is already built to innovate. In addition, we gained experience with teamwork in high stress, competitive environments.
What's next? In the short term, we plan on continuing to build out Communicate.AI to work with a variety of third party apps like Zoom, Teams, etc. Over time we would like to build in functionality for modern hardware acceleration, and begin to integrate Communicate.AI into modern tech stacks.
Log in or sign up for Devpost to join the conversation.