Shader effect applied over original model
Varying degrees of distortion
Shader applied to the slicing attack of a sword

Audio-Based Glitch Shader

Inspiration

The project began as a simple attempt to fuse audio with a visual game element. After taking inspiration from audio visualization graphics and hybrid rhythm games such as Hi-Fi RUSH and _ Crypt of the NecroDancer _, I decided that I wanted to create a unique way of enhancing in-game audio. I was able to accomplish this by combining my computer science background with my fondness for the glitch/digital corruption aesthetic.

What it does

The Audio-Based Glitch Shader, written in ShaderLab and Cg/HLSL, distorts the rendering of a 3D model based on an audio spectrum. By manipulating UV coordinates and offsetting texture sampling, the shader creates a visually captivating synchronization with the audio, producing an immersive and dynamic experience.

How I built it

The project was constructed through a combination of Unity scripting and shader programming. The Unity script, written in C#, interfaces with the audio processing features of Unity to capture the audio spectrum data. The custom shader, implemented in ShaderLab and Cg/HLSL, governs the glitch effect's behavior by distorting the rendering of a 3D model in response to the audio spectrum.

For more info check out my beginner-friendly explanation video: link

Challenges I ran into

The primary challenge for me was simply a knowledge barrier. I'm new to Game development and 3d rendering; and while I had a lot to learn, I was able to use my background in computer science and mathematics to accelerate the rate at which I was able to comprehend how these systems worked.

Accomplishments that I'm proud of

I take pride in achieving my envisioned outcome without compromise or shortcuts. By dedicating time to execute my vision exactly, the final product reflects my creativity and doesn't feel like a product of engineering.

What I learned

The project provided a valuable learning experience, It taught me more about audio processing, Unity shader programming, and the intricate relationship between sound and visuals. Leveraging Unity's shader language (Cg/HLSL) to manipulate visuals in real-time changed my understanding of what's possible in a multimedia format, opening new possibilities for my creative expression.