After hearing about the challenges faced by the multi lingual teams of T-Systems we decided to do something to fix it.
What it does
The project is in two parts:
- First is a web platform where registered users can log in and see their meetings/lectures which have transcribed, search through it by title and content and even watch the live transcription, while taking notes.
- Second is a subtitling system for computers, the microphone listens to the conversation (say in a webex conference call) and transcribes the speech into the language of users choice which is on top of his call, enabling the user to keep focus on the meeting while being able to follow through subtitles.
How we built it
Using the Google Cloud Platform for its Speech to Text and Translate APIs. We set up a streaming voice input from microphone into the cloud speech to text client and transcribe in real-time. This transcribed text is stored and relayed to web app using a flask backend and tkinter is used to show the translation of the stored text as overlaid subtitles.
Challenges we ran into
Resolving dependencies to run a microphone stream for input, using Google Cloud Speech to Text for the first time. Biggest probably was finding a way to non-obtrusively show subtitles on users screen while a call is going on.
Accomplishments that We're proud of
Building a very complete solution in under 24 hours. Team work our team who had never met before showed. Possibly resolving problems for multi-lingual teams and allowing them to show their full potential.
We plan to make the main profit from the sale of available time for broadcasting. It is important to note that immediately upon registration, 2 free hours are given so that users can test and become interested.
Also, it is important to note that in the future plans, the creation of a separate page on which current broadcasts will be displayed and there will be an opportunity to raise your conference as high as possible in order to attract more participants.
What's next for LinguaStream
Do summarization of each meeting/lecture, use natural language processing to auto-generate reports, and enable other devices to act as listeners. Also mobile support for subtitles. In the near future, we plan to add video broadcasts in parallel with the subtitles. Adding a separate chat window for listeners, in addition to communicating with each other, in addition to this, the speaker will see feedback. Creating a universal ranking of speakers that will add an element of competition In the long-term plans, integration with services such as YouTube and Twitch. Also the creation of a separate page on which current broadcasts will be displayed.
The target audience
Companies holding conferences, universities, schools, speakers, anyone who wants to present and capture their report and get feedback.
The closest analogs of the available applications provide only the ability to create subtitles for the voice track, as well as for movies and videos, for example - "Speechnotes". The main difference of our product - is the translation in live time into the selected language and the creation of subtitles for an unlimited number of users.
Why the client will choose us
- Convenient and intuitive interface.
- Service significantly improves the quality and availability of any speech.
- Ability to save your own performances and make notes.
- Expansion of the circle of listeners and visitors.
- Ability to study material regardless of language.
- Convenient interface.
- Download and save heard materials.
- The language barrier disappears.