PyVideoMusicifier

Procedurally Generated Music Videos

PyVideoMusicifier is an application that converts arbitrary input video into a music video based on the accompanying song.

How Does It Work?

There are two main processing techniques used in PyVideoMusicifier, fourier analysis and beat detection.

Fourier Analysis

Fourier analysis allows us to look at the spectral content of the video. From this spectral content we can figure out what the maximum magnitude of all the peaks are. From this magnitude we can adjust the brightness of the video to correspond.

The Fourier transform is also directly visualized in the video. After sampling the transform, the samples are overlayed on the vignette. This allows certain musical effects to be seen as well as heard.

Beat Detection

Beat detection allows us to figure out at what timestamps beats occur in the target audio file. From this list of timestamps we can grow and shrink the vignette effect that is overlaying the video.