Most of the great content for learning is in English, great potential learners could not understand the concepts because of language barrier https://www.nature.com/articles/d41586-019-01797-0. An estimated 25.1 million (41%) were considered Limited English Proficient in United States(https://telelanguage.com/blog/5-statistics-prove-importance-language-support-for-organizations/). Advanced technology is available to bridge the digital divide breaking the language barriers. Motivation for this system are
- As far as we are aware, there is no tool that does a transcoding & transcribing from English simultaneously on existing English content videos, we wanted to build Empowerall on top of AWS services available.
- Content available in the language of choice will enhance learning
- The transcribed content will be highlighted with audio to provide deeper leerning.
- The transcribed content can be printed/ saved as pdf as well The existing close captioning only provide scene by scene captions & does not provide the whole transcript
What it does
The web portal takes a YouTube video/ video file with English content as input, transcribes & translates in to a language of choice, overlaps the translated audio with transcription & presents it to the learner. The transcribed notes could be printed/ saved as pdf.
How we built it
We built a web application using many AMAZON services, S3, Lambda, SQS, AWS Transcode, Amazon transcribe, Amazon translate, Polly, Amazon Mediaconvert. Programmed using Node.js, Python, AWS CLI. UI was built using React.js, material UI
Challenges we ran into
- Identifying the right Amazon services
- Many Node.js libraries were deprecated & we had to find workarounds
- Coming up with the idea & a working architecture with available AWS services which would really help the vision to reduce digital divide
- Merge/ sync the trans-coded audio with original video
Accomplishments that we're proud of
- Contribute global empowerment by breaking the language barriers
- Coming with a solution that has multiple use cases & can be applied to different groups like students, medical professionals, teachers etc..
- The solution is highly scalable & extendible
- Using existing mature AWS services which enables a mature solution to emerge from start
- Relatively cost effective solution
What we learned
- It is possible to come up with a high quality solutions using existing services that are already mature to solve a global problem in a short period of time.
- This hackathon provided us to explore new AWS services that we do not use on day to day basis
What's next for Empowerall
Create a better UI for a more intuitive UX, add supervised learning to include learner feedback , video search both in English & translated language. Create a mobile app for IOS & Android. Make the experience more seamless