Inspiration

While many of us are benefiting from the convenience of taking notes on an electronic device after a long period of remote study, not everyone can afford an iPad or a monitor.

Taking digital notes while watching online recordings on one screen can be frustrating at times when you have to constantly switch between the note-taking tab and video tab. We aim to provide a tool that avoids this clunkiness and ultimately improves student’s productivity.

What it does

It automatically generates a summary of the video and it can also send you a link with the time stamp if you would like to jot down a particular time of the video to go back to in the future. It liberates you from the trouble of manually taking notes and copying the link for the time stamp. The two functions are currently separate but they are expected to be combined in the near future.

How we built it

We build a chrome extension using html, css, and javascript. We embedded a google doc tab into our extension using an iframe element. When the user tries to take notes for a video for the first time, a new google doc page is created, and the URL is stored to be displayed in the extension. This note can be retrieved when the user visits the site again. When the user tries to get the timestamp of the video, a hyperlink at the specific timestamp will be copied into the user’s clipboard for quick pasting into the google doc.

We used a YouTube Transcript API (https://github.com/jdepoix/youtube-transcript-api) to get the transcript of the video. We then used Luhn Summarization algorithm implemented in the Sumy library to pick out sentences in the transcript that best reflect the main topics of the whole video.

Challenges we ran into

We had some trouble finding a proper way to use results of Python functions in JavaScript codes, and that’s why we are having trouble putting the two functions together. We learned a lot about front-end programming, specifically about how to set up a chrome extension, using different APIs, and ways to send messages across tabs.

Accomplishments that we're proud of

We are able to try out different machine learning algorithms and compare their performances tailored to our own needs.

What we learned

We learned about the NLP task of text summarization, how to use an RPC, and much more.

What's next for IntelliNote

Most importantly, we will be integrating the two functions.

Built with: JavaScript, HTML, CSS, Python, Node.js, youtube_transcript_api

Video: Show the process → talk about future steps

Also, right now we are simply storing the mapping between the video URLs and the google doc URLs in a javascript dictionary variable, and this approach may be slow when we have a large number of mappings. In the future, one possible improvement would be to create a database for storing these mappings.

We are also seeking to expand the range of videos we can handle from those with transcripts that have punctuations to those with captions that don’t have punctuations, even to those without captions at all.

Built With

Share this project:

Updates