gptLecture

Summarise your video lecture with AI! Turn long boring video to point-form notes!

Comment

💡 Inspiration

Even though COVID-19 is over, so do the virtual classes nightmare ended, still, sometimes lecturers replace physical classes with recorded lectures. And we all know the real problem of a recorded lectures, it’s LENGTHY AND BORING 🥱 (let’s be honest here). Even if you change the speed to 1.5, it’s still the same, except you can end the suffering faster! At the same time, today’s society is full of ChatGPT noises, why not we harness the power of it? So, what if, we ask AI to listen for us, and then tell us the gist 🤔? This is where our solution, gptLecture comes in!

💭The problem with recorded lectures

  1. Technical difficulties: Technical problems with the recording equipment, unclear voices, or bad quality of the recorded video. 2.Lack of interaction: Recorded lectures may not provide the opportunity for students to ask questions or engage in discussions with the instructor or their peers in real-time, which can limit the learning experience.
  2. Lack of motivation: Some students may find it difficult to stay motivated while watching recorded lectures, particularly if they are not held accountable for their participation or attendance.
  3. Distractions: Students may be easily distracted by their surroundings, such as social media, family members, or roommates, which can impact their ability to focus on the lecture.
  4. Time management: Students may struggle with managing their time effectively when given the flexibility to watch recorded lectures on their own schedule, which can lead to procrastination or falling behind on coursework.
  5. Limited engagement: Students may feel disconnected from the instructor or their classmates when watching recorded lectures, which can impact their sense of community and engagement with the course.

🤨 What is gptLecture?

The short answer is: Using ChatGPT to summarise video lecture for you.

The long answer would be:

  1. Get your video lecture Youtube link. (We assume that most recorded lectures are uploaded to Youtube).
  2. Give the Youtube link to gptLecture. gptLecture will download the video, then process the transcripts from the video.
  3. The transcript will then be processed, and we used prompt engineering to ask for OpenAI’s ChatGPT to perform summarisation of the video.
  4. The summary is in point form, and converted it into a PDF file for you to download!

🫡 How we built it

All of this magic was coded in Python, hosted on Streamlit Cloud. The core of the solution is powered by:

  1. yt-dlp: A library used to download Youtube video, metadata, and transcript.
  2. OpenAI: The API used to communicate with GPT. Specifically, we use the text completion API to summarise chunks of transcripts before combining them all together.
  3. mk2doc: The library used to covert the summarised point into PDF.

We identify the main reason of why the summary is so accurate and powerful. We perform iterations of prompt engineering in order to get the most accurate summary. For example, simply invoking GPT with “Summarise the text below: [the text]” will give a less accurate result. However, if the prompt was “You are currently summarising Lecture 1: Calculus. Here is the transcript of the for the first chunk of the lecture: [the text]” produced a better result.

😣 Challenges we ran into

Initially, the full chunks of video transcript is passed into the OpenAI’s GPT API, but the request failed since it reached the maximum number of characters allowed. Therefore, the transcript is split into chunks of text, and let the API to process them one by one. The chunks are then combined together to produce the final notes.

The problem with Youtube video auto-generated transcript, especially those video with strong accents. The transcript may give wrong details, but thanks to GPT’s feature of auto-correcting it, we found that most of the time, the wrong/ambiguous sentences get corrected automatically.

Lastly, the OpenAI’s API free tier limit. This API does not allow frequent call, which will result in being rate-limited.

🫶 Accomplishments that we're proud of

In less than 10 hours, and merely 200 lines of code, the initial MVP is ready! Thanks to Streamlit Cloud and Github Codespace for generously providing free resources to allow testing and hosting final MVP.

🤞What's next for gptLecture

Imagine this. You bring your phone to physical classes. After the class finishes, we provide you a summary of the class, wouldn’t that be cool? Yes, this is what we will be doing after this hackathon!

Built With

Share this project:

Updates