When our institute shifted to work-from-home mode, a lot of our peers had to attend classes from remote parts of the country, where they didn't have access to high-speed internet, and thus faced a lot of problems in streaming class videos. We also felt that our proposed solution could be used to ease the learning process of all the other students as well, as we describe below.
What it does
Our application provides an interface for professors to upload lectures for their courses, and students to register for the class and access the lectures. Sounds simple right? Our major innovation arises from our feature where, given the lecture video and slides, we process and extract audio and subtitles and for each slide separately. This allows those with low speed internet access to read the subtitles for the lecture, or listen to the audio file. It is also easier for everyone to revise their content.
How we built it
We use flask for the web app. We have used OpenCV for detecting slide changes in the lecture video, and the google cloud API for speech-to-text for extracting the subtitles.
Challenges we ran into
Our biggest challenge was working and collaborating remotely from our homes. We also faced a major challenge in detecting slide changes via image matching. We tried feature extraction, histogram and structural similarity, which didn't work. We finally used OCR in the end which worked!
Accomplishments that we're proud of
For working and getting so much running in such a short time!
What we learned
It was the first time we were working with Flask, the google cloud API and so many frameworks for image matching. We learnt a lot about them.
What's next for SmartSlide
We'll continue to add more features such as a search bar, more links for navigation, etc to improve the user experience.