We have a diverse friend group that we managed to form during the in-person school year, but many people do not have the opportunity to do so. The extensive use of Zoom as well the expansion of global institutions inspire the need for global communications.
What it does
It's a zoom conference app, but you hear the translated audio of your language of choice, for everyone in the call.
How I built it
We built the backend in the Spring Java framework and the front-end in ReactJS. We used the WebRTC protocol for video streaming and WebSocket in the backend. The translation and audio synthesis are all from the Azure libraries that Microsoft developed.
Challenges I ran into
We could not get the WebSocket to establish connections, and we needed to proxy the front-end to back-end requests. The API also took a while to figure out and process given the compatibility of Spring and ReactJS.
Accomplishments that I'm proud of
We built a good login system and managed to test the API fully. The video chat also partially works.
What I learned
We learned a lot about ReactJS and the protocols for peer-to-peer connections.
What's next for Dubbed Zoom
Full integration of everything and a built-in voice synthesizer to produce your voice for other participants.