LecBuddy

Inspiration

As university students, we watch lecture recordings on a day-to-day basis. However, there are numerous hassles which we may face when doing so. Not only are lecture recordings usually so long in duration, but not all parts of the recording are useful for learning. For instance, the professor might digress to other topics in the middle of the lecture. Moreover, when reviewing lecture recordings for revision purposes, it is difficult to remember which parts of the lecture contains the key topics and finding them manually takes time.

Therefore, we have proposed LecBuddy as a solution. LecBuddy aims to help students reclaim their time. By transforming and summarizing long-form video content into structured, searchable text, it allows for concise walkthrough, efficient revision and quick referencing of complex lecture material.

What it does

LecBuddy first detects a Panopto lecture. It then detects whether the lecture has captions attached to it or not. If it does, then it intercepts the caption from Panopto API and uses them to summarize the lecture content into the relevant key topics and their bullets, along with the timestamps. If the lecture does not have captions attached to it, then LecBuddy would perform audio extraction and transcription, then summarize the transcription to once again generate the key topics and timestamps. Note that users can click these timestamps in the extension to navigate to the exact timestamp in the lecture video. LecBuddy would then display the lecture summary and transcript to the user. The user could choose to download the lecture summary in the form of a markdown file, and download the lecture transcript in the form of a pdf file.

How we built it

We utilized React and Tailwind CSS to develop LecBuddy's frontend interface. Aside from that, we also used Chrome extension APIs for browser interactions. For the backend side, we used Node.js and Express. Furthermore, we used additional tools such as FFmpeg for audio extraction, OpenAI Whisper-1 for audio transcription, and Gemini-2.5 Flash for summary generation. Lastly, for deployment and CI/CD, we utilised Docker and AWS.

Challenges we ran into

We faced numerous challenges throughout this hackathon. The most prominent of which was with our audio extraction and transcription pipeline. The audio extraction process using an unfamiliar tool, FFmpeg, was difficult. Also, extracting audio and transcribing it from a typical lecture takes a long time, and we initially struggled to make the process more efficient. Eventually, we were able to make it faster by sequential slicing into 10-minute chunks and transcribing each chunk in parallel before combining them for the final transcript.

Other than that, we also initially quickly ran out of tokens using our free tier models, but careful chunking and the free credits provided helped us solve this issue.

Accomplishments that we're proud of

We are happy that we are able to build a functional prototype of our application within one day, which includes all the features to make this app usable. Doing this project within this time constraint also has truly helped us achieve a level of discipline we have never reached before, and so we were able to work hard enough to deliver a working product. Throughout this project, we were also happy to get more understanding of state-of-the-art tools used in the software engineering industry.

What we learned

Through this project, we learned to develop a Chrome extension and using its API for browser interactions. Developing a Chrome extension improved our proficiency to debug with DevTools, which included intercepting files through Networks, and looking at the Console output. We also gained more proficiency with frontend tools (React, Tailwind CSS), backend technologies such as Node.js and Express, and deployment tools such as Docker and AWS. Finally, this project also helped us gain exposure in utilising LLMs for our applications, and helped us learn how to carefully work with the rate limits that the LLM APIs have.

What's next for LecBuddy

Summary improvements: More accurately summarize the key topics in a lecture
Frontend interface improvements: Minor frontend interface improvements
(Additional feature) LLM query feature: For users to query if they have any enquiries about the material
Persistent storage: Store all previously user-generated lecture summaries for their future reference