Vid2Notes

Quickly convert long-form videos into condensed notes and flashcards through the use of AI, allowing one to study much more efficiently. Excellent for students studying math and the sciences!

Comment

Inspiration

As people who value academics very highly, our team’s members have always wanted a way to optimize time and learning. When the hackathon themes of AI/ML and education were announced we were ecstatic because it was a huge opportunity to pursue a growing interest in AI/ML while giving us a new tool that could help us and many others towards their academic goals. We ultimately decided to build a service that summarizes YouTube videos due to how important online video has been to our learning, especially since the pandemic. We chose to build a web interface to serve our app, as we believed it would bring the most impact to the most people due to how accessible the web is.

What it does

Vid2Notes is a web tool which scrapes any YouTube video (that has a transcript) and outputs a summary in point form for the user to easily understand the video without having to watch it. It also has an option which allows the user to "study" using the Flash Card function, similar to the way Quizlet works. In addition, this can also be used for those who have hearing disabilities, or are in settings where they simply cannot use audio.

How we built it

  • After realizing we would be converting video to short point form, we realized our backend needed two main components
    • One which obtained the video and put it into a form that can be analyzed quickly
      • We figured using plaintext would be the quickest to convert and analyze
    • The other part needed to convert the plaintext of the video into something that can be useful to students, such as notes or flashcards
      • We realized this would be an excellent use case for modern AI such as ChatGPT
  • As aforementioned, we already knew we wanted to serve the results of our scripts on the web because of its accessibility and large potential impact on potential users
    • We chose to work with Next.js and Django because our team was already familiar with these web frameworks

Challenges we ran into

Firstly, one issue we found was a way to actually analyze the transcript, and summarize it. Due to the time limit, we decided to look for modern AI API's that would be able to summarize a text, whilst being free to use and not being too limited in how often it can be used. Eventually, we managed to find one that was free and could be used as much as needed, called Cohere, which gave free API keys that had a reasonable rate limit time and maximum request length.

Accomplishments that we're proud of

This was the first Hackathon all of us have participated in, so we are very proud that we managed to complete a substantial prototype during the 24 hours of the hack. Going into the Hackathon, we each had low expectations on what we could do, but to actually produce something tangible at the end was very fulfilling.

What we learned

Throughout this Hackathon, we have learned many things:

  • How to structure a project with many moving parts (making sure the frontend, backend and server are all fully compatible)
  • Proper Github usage whilst working with a team
    • Handling merge conflicts
    • Being able to integrate various code segments together, which were worked on by different people
  • Researching how to thoroughly use external APIs?
    • How to integrate APIs
    • Limitations of limited access to APIs
  • The blessing of sleep

What’s next for Vid2Notes

There are many ways we can further improve Vid2Notes in the future. Below is a list of the main ways we want to make Vid2Notes more useful:

  • Using a method of converting speech to a transcript, rather than relying entirely on YouTube -made transcripts (which may be inaccurate, or not even exist for some videos)
    • This, alongside potentially transcribing the significant visual components of the video, may allow for more accurate condensing of notes
  • Noticing when a keyword is being used (such as “Continuity” in a Calculus video), and showing its definition when requested
  • Visualizing some of the key topics being shown
    • For example, if a function f(x) = sin(x)/x is mentioned, showcasing a graph of that function on the same page would be very useful for ones notes
    • This could be done by getting some snapshots of the visual components, as oftentimes the key visuals are presented in the video at the same time as it is being explained verbally

Built With

Share this project:

Updates