Since the onset of COVID-19, most of the classes have been taking place remote/online. This shift in the mode of delivery has made education less personal , more susceptible to technical difficulties, and the worst of all- more unequal.
Unlimited Internet connectivity or great bandwidth is not a trivial matter to some. . If ever there is a connectivity issue, live lectures is not an option. Even if students have access to those resources, some students are not meant to learn by focusing for an arduously long time on a screen. Classroom lectures can be as long as 3 hours for several asynchronous courses. The Federal Communication Commission estimates that 19 million Americans do not have access to a fast internet connection Article. No child’s education can be an afterthought, and we knew that the current landscape could use some improvement.
Looking at online education sites like Coursera, EdX, etc. we found our spark- they provide transcripts of the video lectures which are immensely helpful for students to follow the lecture even when the videos do not stream well ! Even after working around the bandwidth problem, several more lied ahead of us
- Lectures overwhelm most children, but multiple hour long videos on a screen everyday is just a non starter for children with special needs, ADD or ADHD.
- Students often struggle with time management and tend to pile up work inevitably creating an insurmountable set of backlogged videos before an important semester examination.
- Obtaining a concise summary of video lectures centered on relevant concepts is worth its weight in gold. It is a tool of revision for students that have access and a saving grace for students struggling to understand what is being taught or for students who, through no fault of their own, simply lack good internet infrastructure. A compiled summary of lectures conducted throughout the semester can help almost every student prepare. It can serve as a prerequisite course helping teachers with establishing concepts or as a handbook aiding budding professionals with interview prep. It requires considerably less bandwidth/internet connection speed than watching video lectures.
What it does
EduAid is a platform for students to obtain short relevant summaries of long video lectures. It allows professors/teachers to use the platform to upload recorded video lectures. As video lectures are uploaded, the platform automatically generates a short text summarization for each video and stores it. Students using the platform are able to view both the uploaded videos and the auto generated text summarizations. This gives students a brief overview of the concepts introduced in the video, allowing them to further research the concepts through articles/blogs online as well as follow along with the videos more easily.
How we built it
EduAid is a web application built with Flask in Python3. The data is hosted on a sqlite database. Audio is extracted from the uploaded videos using ffmpeg. The audio is converted to speech using Silero Speech-To-Text models. The extracted text is then summarized using pretrained models built on top of BERT. The paper and their implementation of the model.
Challenges we ran into
- Video upload and storage with Flask
- Adapting research code with our web app
- PyTorch framework issues with loading pretrained models
Accomplishments that we're proud of
- Reading state of the art research papers in text summarization and utilizing it for our application
- Building an end to end application with deep learning to address a modern problem.
- Developing a web application end-to-end with database design
What we learned
The following are our takeaways from this hackathon
- State of the art NLP models (Transformer, BERT) and the current research into text summarization
- Pretrained models in PyTorch for speech to text conversion
- Deployment of deep learning models with webapps built with Flask framework.
- Database design for business applications
- Formulating a business problem and using deep learning to find solutions for it in a short time period.
What's next for EduAid
We envision extending EduAid into a comprehensive platform that handles all logistic aspects of online education. Some of the ways that EduAid can be extended include
- Parse the summarized text and link it back to the corresponding section(exact time) in the video
- Allow assessments such as quiz, exams to be added to each of the courses
- Perform grading on the assessments and data analysis on the performance of the students to help with continuous improvement
- Dockerize and deploy the web application on a cloud platform like Heroku/GCP.