Music March

• 

  Screenshot of our audio visualizer

• 

  Screenshot of our initial submission to the huawei challenge - running in real time

Inspiration

From our curiosity in graphics and interest in electronic music, our idea to make a vivid and interactive 3D audio visualizer was born!

What it does

Beyond our submission for "Huawei's challenge 1 - Ray Marching", we built our visualizer to take in audio from a source, run frequency analysis, and translate the bass, middle range, and treble into a usable coefficient to then amplify the height map of different hilly structures around the user. A user may also choose to navigate through the environment, admiring up close or from afar the beauty of our infinitely generated terrain

How we built it

While we initially based our project on an SDL3 implementation of the Huawei challenge 1 shader, our ambition lead to a complete overhaul in the fragment shader. Our pipeline is as follows: Audio Input -> Fast Fourier Transform -> Aggregation and Normalization of Bass, Mid-range, and Treble Frequencies -> GPU Buffer -> Fragment Shader -> Ray Marching -> Terrain height generation (Fractal Brownian Motion + Audio Frequencies) -> Rule based Coloring

Challenges we ran into

During our time with Huawei's challenge, we found it very difficult to implement the SDF, and settled for some other meaningful optimizations, leading to at least a 20x increase in frame rate performance, starting with our laptops running the ray marching algorithm at 3 fps, and at the end, 60 fps with little variance.

Accomplishments that we're proud of

Beyond our optimization results, we are super proud of how well the audio visualizer appears after our work through the night

What we learned

We learned a lot of valuable computer graphics techniques, as well as the difficulty of procedural generation when using techniques such as Ray marching

What's next for Music March

We plan to extend and improve the visualization techniques, and in the near future, use it for background visuals while DJing

Built With

• cpp
• glsl
• sdl