We spend a noticeable amount of our weekly time watching YouTube videos, be it for entertainment, education, or exploring our interests. In most cases, the overall intent is to obtain some form of information from the video. We were seeking a solution to increase the efficiency of this "information extraction" process as YouTube's speed adjustment option is the only relevant tool. And so we decided to develop YouTube Summarizer!
What it does
The summarizer is a Chrome extension that works with YouTube to extract the key points of a video and make them accessible to the user. The summary is customizable per user's request, allowing varying extents of summarization. Key points from the summarization process, together with corresponding time-stamps, are then presented to the user through a small UI next to the video feed. This allows the user to navigate to more important sections of the video, to get to the key points more efficiently.
How I built it
We first set up a GitHub repo for project management and made a readme to keep track of dependencies and environment information. Our project utilized a Google Chrome extension for the frontend and a Django server for the backend, so we initially developed both parts of the project separately and integrated each service towards the end of the hackathon. We also had to choose which online APIs and services to use as our project progressed, and decided to use Punctuator, Resoomer, and YouTube closed captions as the key technologies involved in the timestamped summary generation.
Challenges I ran into
Out of the numerous challenges we encountered throughout this hackathon, the most significant challenge was an overestimation of the technologies available to us. Our overestimation of the ability of transcription services forced us to make compromises, while our attempts to get the YouTube player to control playback pressed us to create a hacky solution. Overcoming these challenges and bridging the capabilities of these technologies was an integral part of the project.
Accomplishments and What I learned
What's next for YouTube Summarizer
Even though our current version of Youtube summarizer is able to provide users with valuable information regarding the videos they watch and can provide a compact summary of the video, we believe that this tools can be further developed to meet the needs of YouTubers. While we used advanced online services to provide a summary of the description, we believe that we can use a variety of text, audio and video analysis tool to provide valuable information and more accurate summaries for videos. As an example, we attempted to use the online Google Cloud AI services to provide transcripts for videos with no captions and use AWS services to analyze audio files and figure out important sections of the video based on its correlated audio file. Unfortunately, many of these services required more time in order to fully analyze the information and were not incorporated into the final product. In addition, we believe that using Youtube’s video API, we will be able to provide more control for our users and give them more freedom when using this tool. Overall, we believe that video analysis and summarization is not only important to the individual, but also proves to be more important when considering big data analysis. Hence, we believe the youtube summarizer can be the start of a novel technology in informatics.