Inspiration

The inspiration for this app is from our personal experience through talking to the elderly over video call. Many elderly people have deteriorated hearing, and experience difficulty hearing voices over video call. We thought that allowing conversations to be read would greatly benefit this group of people. Furthermore, we thought that this could be applied to education as it would be possible to record tutoring sessions over Skype or learn a new language through translation of the saved text.

What it does

This webapp makes use of a Google cloud API to transcribe spoken words into written text. Working almost like creating subtitles for video calling, it allows users to live stream sentences in a log box. Furthermore, this webapp saves the conversation in a txt. file, allowing users to access it later or share it with others.

How we built it

We combined two different APIs, the Google cloud speech to text and the Twilio Real-Time WebRTC video infrastructure to create our web app. We used Flask for the backend and voice-to-text, and HTML/CSS/Javascript for the frontend.

Challenges we ran into

We ran into a lot of trouble getting the front end and the back end to communicate in real time. We attempted rapid HTTP requests, web sockets (sockets.io), and text-file transfers to transfer data from server to client. Subttle eventually settled on sending text files between the server and client for speech-to-text.

Accomplishments that we're proud of

We're really proud of the way we build the backend and frontend of the apps. None of us knew much about web dev beforehand, and we were excited to work on a project like this.

What we learned

We learned A TON about web development that we haven't had the chance to learn in college - including how to use APIs, how websites are structured, and how servers and clients communicate.

What's next for Subttle

We want to fully implement speech-to-text in real time, as well extend our services to the education sector using translation capabilities of Google's API.

Share this project:
×

Updates