LectureLeaf

Inspiration

LectureLeaf started from a simple problem I faced while studying from lectures on my tablet.

Whenever I was watching a lecture and came across an important slide, diagram, or explanation, I would pause the video and take a screenshot. After doing this multiple times, I would end up with a lot of screenshots that I had to sort through later. Since I study on my tablet, constantly switching between the lecture, screenshots, and notes also made the whole process feel unnecessarily slow.

I wanted a way to save the important parts of a lecture without constantly interrupting my learning. That is what inspired me to build LectureLeaf.

What it does

LectureLeaf helps students turn video lectures into organized study material.

Instead of manually pausing a lecture and taking screenshots every time something important appears, LectureLeaf is designed to automatically capture the useful parts of a lecture as precisely as possible.

The idea is to make the process simple:

Watch → Capture → Organize → Study

This makes it easier to revisit important concepts and study from the captured material without having to go through a large collection of random screenshots.

How I built it

I built LectureLeaf completely on my own during the hackathon.

A major part of the project was figuring out how to identify the right moments in a lecture and capture screenshots with as much of the relevant information visible as possible. I used OpenCV for video frame and image processing and worked with the Whisper model for understanding the lecture audio.

I also wanted the project to be usable without depending entirely on API keys or an internet connection. Because of that, I explored offline alternatives and fallback modes so that the application could still work when an API-based approach wasn't available.

A lot of the development involved experimenting with timing, frame selection, and different ways of determining when the important information on the screen was actually visible. The goal was to avoid simply taking screenshots at fixed intervals and instead make the captures as useful and precise as possible.

Challenges I ran into

One of the biggest challenges was deployment.

The YouTube video processing works correctly when I run LectureLeaf locally, but I haven't been able to figure out why the same YouTube videos are not being processed when using the deployed version. This was especially challenging because the same workflow works as expected in my local environment, making it difficult to identify what is different in the deployed environment.

I spent a significant amount of time debugging this and trying to understand why the behaviour changes between the local and deployed versions. I haven't completely resolved this issue yet, but it taught me a lot about how differences between development and deployment environments can affect a project.

Another challenge was getting the screenshot timing right. Since lectures can move quickly, capturing even slightly too early or too late could mean missing part of a slide or capturing it before all the important information was visible. I spent a lot of time experimenting with frame selection and timing to make the screenshots as precise as possible within the limited time of the hackathon.

Accomplishments that I'm proud of

The biggest accomplishment I'm proud of is that I was able to build the entire project on my own within the limited time of the hackathon.

More importantly, I was able to get the screenshot capture as precise as possible, while still making sure that the important information on the screen was visible. Getting that balance right took quite a bit of experimentation, especially because lectures don't always stay on one slide for the same amount of time.

I'm also proud that I was able to take an idea based on something I personally found annoying while studying and turn it into a working project within the hackathon.

What I learned

This project helped me learn much more about OpenCV and working with video frames and image processing than I had worked with before.

I also learned more about the Whisper model and how audio and transcription can be used as part of a larger workflow instead of treating them as standalone features.

Another important thing I learned was the value of having offline modes and fallbacks. Instead of designing the entire application around API keys, I explored ways to make parts of the application work locally as well. This made me think more carefully about reliability and what happens when an external service isn't available.

I also learned a lot about deployment and debugging issues that don't appear when everything is running locally.

Most importantly, I learned a lot from taking the project from an idea to a working application completely by myself within the time constraints of the hackathon.

What's next for LectureLeaf

For now, I mainly want to focus on making the screenshot capture as precise and reliable as possible.

I also want to figure out why YouTube video processing works locally but not on the deployed version, so that the complete workflow works reliably outside my local environment.

Once these core parts are stable, I can explore additional features around organizing and using the captured material for studying.

Built With

• css
• flask
• html
• javascript
• opencv
• python
• react
• render
• typescript
• vercel
• whisper
• yt-dlp