Cosmic Hum

The most intuitive musical instrument we possess is our own voice. Cosmic Hum was created to prove that technology can turn this amazing tool we have into a direct creative art form.

Comment

Inspiration

Art and music have almost always needed a physical tool of some kind, a paintbrush, a keyboard, a mouse click. But the most intuitive instrument any of us carry around is our own voice. I wanted to explore what happens when humming, something everyone can do without training, becomes the direct input for both a visual artwork and a piece of music. That question led me to Hack The Arts 2026's theme of building art that couldn't exist without technology, and Cosmic Hum became my answer. I wanted a project where the technology wasn't decoration on top of the art. The pitch detection and the emotion inference had to be the actual paintbrush.

What It Does

Cosmic Hum is a browser-based synesthetic art studio. You grant microphone access and start humming. As you hum, the app listens to your voice in real time, tracks your pitch using a custom autocorrelation algorithm, and optionally snaps every note to an in-tune musical scale. Hence, a shaky hum still comes out sounding musical. At the same time, an on-device neural network estimates the emotional prosody in your voice. All of that data feeds into a visual layer where abstract shapes bloom continuously across a starry canvas, their color and form driven by frequency and their density driven by amplitude, while a Simplex noise particle field adds ambient motion around them. The frequency-to-color mapping is a simple hue rotation $$ \text{hue} = \frac{(f - 80) \bmod 180}{180} \times 360 $$ so every 180 Hz the color cycle resets, giving the canvas a recognizable rhythm even as new shapes keep appearing. Once you are done humming, you can press play and hear your creation synthesized back with polyphonic oscillators, an ADSR envelope, and concert hall reverb. You can scrub through the timeline, pause and resume, switch between real time and condensed rhythm playback, and turn on four-way kaleidoscope symmetry to fold your melody into a mandala. When you are happy with it, you can export a PNG snapshot, an animated GIF, a standard MIDI file for use in a DAW like Ableton or Logic, or a compressed shareable URL.

How We Built It

Everything runs one hundred percent client-side, no backend or server of any kind. The app is built with Vite and TypeScript in strict mode for fast iteration and reliable types. Audio capture and analysis go through the native Web Audio API, using an AnalyserNode for FFT and time domain data, feeding a custom autocorrelation pitch detector that applies parabolic interpolation for sub-sample accuracy and an exponential moving average to smooth the output. Musical playback uses OscillatorNode instances with a full ADSR envelope layered under a DynamicsCompressorNode and a reverb network for a more polished, studio-like sound. The emotion classifier runs locally through Hugging Face's Transformers.js using a quantized ONNX wav2vec2 model, buffering about a second and a half of resampled audio before classifying happy, sad, angry, fear, neutral, surprise, or disgust, with a prosody-based heuristic fallback in case the model fails to load. On the visual side, everything is drawn on HTML5 Canvas 2D. There are over sixty unique shape types spanning geometric, organic, and abstract parametric families, chosen through frequency-mapped buckets and rotated through an emotion-aware pool. Rendering uses viewport frustum culling and careful context state management across the mirrored kaleidoscope quadrants to keep the whole thing running at sixty frames per second even with hundreds of shapes and an active particle system on screen. Sharing works by serializing notes to JSON, compressing them with CompressionStream and gzip, and encoding the result into the URL, capped to the most recent notes so links stay a reasonable length. A local gallery saves thumbnails and note data to localStorage so past creations can be reopened later.

Challenges We Ran Into

The biggest challenge was performance once the canvas started filling up with hundreds of overlapping shapes, live rotation, an active particle system, and four-way kaleidoscope symmetry all running at once. Early builds dropped frames badly under that load. We solved this with spatial culling so shapes outside the current viewport never get rendered, and by optimizing how canvas context state gets saved and restored across each mirrored symmetry quadrant so we weren't repeating expensive operations four times over. Getting the pitch detector to feel musical rather than technically correct was its own challenge too. Raw autocorrelation output is noisy and jumps around in a way that looks fine on a graph but sounds and looks jittery in an art piece, so we spent real time tuning the smoothing and the scale snapping so a wobbly human hum still produces something that feels intentional.

Accomplishments We're Proud Of

We are proud that Cosmic Hum achieves essentially zero-latency voice-to-visual translation entirely inside the browser with no backend at all. Pairing that with studio-quality reverb-heavy playback and genuine MIDI export took this past a fun browser toy and into something that could actually feed into a real music production workflow. Getting a neural emotion classifier to run smoothly on device, with a graceful fallback when it can't load, was also something we're happy to have pulled off inside a hackathon timeline.

What We Learned

Building Cosmic Hum deepened our understanding of digital signal processing, how the Web Audio API's context and node graph actually behave under load, and how to keep a Canvas render loop fast when it needs to draw a large and constantly growing number of objects every frame. We also learned a lot about running machine learning models directly in the browser, from quantization tradeoffs to the practical realities of buffering audio for a model that expects a fixed window size, and about designing a sensible fallback path for when that model isn't available.

What's Next for Cosmic Hum

We would like to expand Cosmic Hum into a collaborative jam room where multiple people can hum into the same canvas together over WebSockets, turning it from a solo synesthetic sketchbook into a shared instrument. We are also interested in exploring a WebXR version that places the blooming shapes in true 3D space around the listener, with spatial audio that matches the visual depth.

Built With

Share this project:

Updates