Example output using the iphone10 ad can be seen here: https://drive.google.com/file/d/1mI32xOBVSMVU9EPXqrc1a6MzDn6sZFZM/view?usp=sharing It was a 5:30 video, we summarized it into a 1:00 highlight video!
Videos could be long and tedious, so we created an automatic routine to capture the highlight of videos.
What it does
Capture the highlights of a long video using audio clues, then concatenate the highlights into one short video. A streamline of the procedure: Input a long video -> capture the audio file -> analyze the audio file to produce an array of energy level for each 50ms -> calculate the clusters of high-energy intervals according to the array -> produce an array of time stamps for such intervals -> concat the subclips into a highlight video.
How we built it
Use PyAudioAnalysis to calculate the energy level for each 50ms Use this information to iteratively find the best clips of the video The rules are: no more than x low energy blocks; at least y high energy blocks; at most z percent low energy blocks in each clip concatenate the clips into one highlight video
Challenges we ran into
An ideal highlight clip is hard to define and hard to implement
Accomplishments that we're proud of
We successfully quantitatively defined a highlight clip. We successfully programmed the logic to find such ideal highlight clips Using SVM models from GitHub, we analyzed the emotion and distinguished speech from music, which can be extended into a better product in the future.
What we learned
Learned to use python to effectively do video analysis and editing.
What's next for Audio-driven Automatic Video Highlight Capture
Could implement a User Interface; Reduce manual procedures; Could explore more functions we can achieve using the strong tools;