Mathverse — Project Story
Inspiration
Most people meet the Mandelbrot set, the Lorenz attractor, or Chladni's vibrating plates as a single static image in a textbook or a YouTube thumbnail — beautiful, but frozen. The equations behind them aren't frozen at all; they're dynamical systems that unfold over time, and several of them (Chladni plates, hydrogen orbitals) were discovered through sound in the first place. That felt like the actual idea worth building: instead of another gallery of pretty fractal renders, an instrument where the math, the visuals, and the audio are all the same live system — turn a formula, hear it change, watch it change, at the same time.
(This is the spot to add your own line about what specifically pulled you toward "Hack the Arts" and this project — what you'd seen or wanted to make playable. That personal beat is worth writing in your own words.)
What it does
Mathverse is a real-time, browser-based instrument spanning seven mathematical "realms" — Fractals, Strange Attractors, Cosmic Nebulae, Density Fields, Cymatics, Curves, and Number Fields — with 73 built-in presets total, each credited to its real discoverer where known (Mandelbrot, Lorenz, Chladni, and others). Nothing is pre-rendered: every fractal is compiled into a live GLSL shader, every attractor is numerically integrated in the browser, and every parameter — from a Julia constant to an attractor's differential coefficients — can be retuned live with a slider or a keystroke, with the sound engine reacting in the same instant. A-through-L on the keyboard play notes while simultaneously reshaping whatever system is active; a "Surprise Me" roll jumps to a random universe across all seven realms with its own arpeggio; a Jam Mode lets the instrument compose and perform on its own. Every session can be captured as a PNG, recorded as a WebM clip with live audio, or shared as a single URL that fully reconstructs the scene — no backend involved.
How we built it
The rendering layer is Three.js over WebGL. Fractals are handled by compiling the user's formula (e.g. cmul(z,z) + c) into a GLSL fragment shader on the fly, built on top of hand-written complex-number helpers (cmul, cdiv, cpow, csin, ccos, …) so arbitrary complex-iteration formulas can be typed in and rendered immediately. The other six realms are point-geometry systems computed on the CPU each time a parameter changes — attractors are integrated step-by-step (RK2), density fields and nebulae iterate tens to hundreds of thousands of points per frame, and curves and number-field formulas are parsed by a small hand-rolled math expression compiler that tokenizes and recursively parses expressions into real JS functions — deliberately built without eval, both for safety and because it made runtime validation of user formulas possible.
The audio side is a custom synth built directly on the Web Audio API with four selectable scales (pentatonic, lydian, phrygian, whole-tone), including a granular mode that samples a fractal's own escape-time values and plays them back as grains — so the visual math literally becomes the sound texture. The whole thing ships as a single-page React + TypeScript app built with Vite, and vite-plugin-singlefile bundles the production build into one self-contained HTML file, so it runs with zero server or setup for hackathon judging.
Challenges we ran into
- Letting people type arbitrary formulas without using
eval. Both the shader-formula path (fractals) and the expression path (curves, number fields) accept free-form user input, which meant writing a real tokenizer/parser for the expression compiler and a validation pass (validateFragment) that catches a broken GLSL formula before it's handed to the WebGL context — an invalid shader can silently fail or crash the render loop, so bad input has to be caught and reported back as an error message instead. - Keeping seven structurally different systems feeling like one instrument. A shader-based fractal, an ODE-integrated attractor, and a point-cloud density field are computed in completely different ways under the hood, but from the keyboard they all had to respond the same way — a note press has to morph whatever system is currently active, which meant designing a shared interaction layer on top of very different rendering pipelines.
- Making Jam Mode sound musical instead of random. Auto-composing mode fires notes on one randomized interval (400–800ms), modulates parameters on another (2–3s), and switches presets on a third (20–40s) — tuning those three independent timers against each other so the result felt like a performance rather than noise took real trial and error.
- Performance at scale. Density fields push up to 800,000 points and nebulae several hundred thousand more, all recomputed live as sliders move — keeping that interactive in a browser tab meant being careful about what got recalculated on every parameter change versus only on preset switch.
Accomplishments that we're proud of
Building a real (if intentionally small) shader compiler and a real expression-language compiler from scratch, rather than reaching for eval or an existing formula-parsing library. Getting seven mathematically distinct domains — complex-plane iteration, ODE integration, point-cloud attractors, acoustic wave functions, prime-number geometry — under one consistent, keyboard-playable interface. And shipping the entire experience, including recording and URL-based state sharing, with no backend, no database, and no API keys at all.
What we learned
A hand-rolled recursive-descent parser is much more approachable to build than it sounds, and it's meaningfully safer than eval the moment you're accepting user-typed formulas. Treating sound and visuals as one system driven by the same state — rather than bolting audio reactivity onto a finished visualizer afterward — is what actually made the "instrument" framing work; retrofitting that sync later would have been far harder than designing for it from the start. And real-time WebGL performance work is as much about deciding what not to recompute on every frame as it is about the math itself.
What's next for MathVerse
- MIDI output, so the instrument can drive an external synth or DAW instead of only its own built-in sound engine.
- More realms — cellular automata and L-systems are natural next additions to the existing seven.
- A shared preset gallery, so people can publish and discover new formulas and systems the way they can already share a session via URL.
- Collaborative jam sessions, letting more than one person shape the same live system together.
Log in or sign up for Devpost to join the conversation.