Due to COVID-19, millions of students across the world have been forced to quickly adapt to video lectures and online education. To ease this transition and make studying more efficient, we wanted to help students by summarizing their lecture transcripts while also capturing the full lecture transcript so students can pay full attention to the lecture instead of being distracted with taking notes.
What it does
A website that summarizes video lectures for efficient studying. Users can upload their video lectures to receive a full text transcript and timestamps of the most important sentences. The video timestamps have not been formatted to "minutes : seconds" yet.
How We built it
We made made a microservice out of a pre-trained BERT model to summarize text, and an Express web server that works with Vue for the UI to make a web app. The web app accepts video uploads, sends the video to Azure's Speech-to-Text API to get a full transcript, sends the transcript to the microservice to get a summary with timestamps, and sends the summary and timestamps to Vue for display and video playback.
Challenges we ran into
Managing cloud platform credentials in a team of 4 was difficult. Coordinating ourselves to avoid duplicating work. Managing packages and dependencies. Scope creep. Timestamps need to formatted to "minutes : seconds".
Accomplishments that we're proud of
Most technically sophisticated hackathon project so far, as the project has many moving parts like Azure Media Services (for video playback), Azure Cognitive Services (for the Speech-to-Text API), and BERT (for text summarization). Fun name ;)
What we learned
What's next for AweSummarizer
Adding video lecture subtitles automatically generated with Azure Media Services. Adding clickable timestamps to play the video at that timestamp for more convenient navigation.