Covid-19 has disrupted education and most classes are now online. However, it is very time-consuming to review long lectures and find the relevant content you want to study.
What it does
Smart Class allows students to easily review lectures by automatically transcribing the recordings, organizing the content, and adding insights in a user-friendly interface. Here is a summary of the main features:
Transcription and Search
Students can easily review lectures transcriptions and start playing from any point they want to. It is also very easy to find relevant parts by using the integrated search feature.
Relevant keywords are extracted using Natural Language Processing and links to Wikipedia are added for each keyword in the transcription so that students can find additional learning material.
Messages are grouped by topics so that students can focus on studying specific parts they want to improve on.
Students can search for which questions were asked either by the lecturer or the students.
Action items and Follow-ups
Students can also see which actions and follow-ups need to be done, such as project deadlines or homework they need to finish on a certain date
The lectures are separated by each speaker so that students can review content separately.
How we built it
- Student pastes lecture link
- Video is downloaded
- Video is converted to mp3
- Symbl Audio Async API is called using the processed mp3 file with diarization enabled
- The lecture description, title, and the
jobIdare saved in a MongoDB database
- The job status is monitored until it is completed
- After the job is completed, the following information is extracted:
- Conversation info
- Transcription (messages)
- Action items
- Keywords are extracted using NLTK
- Links are added to messages containing the keywords
- Messages are grouped by speakers
- Messages are grouped by topics
- The timestamp for each message is computed using the conversation start date as a reference
- Context messages are added for actions, follow-ups and questions
What's next for Smart Class
- Allow more media types
- Add more related content
- Add inline related content