Sound Matter Engine

An AI platform that transforms music into unique 3D sculptures. Every sound has its own sonic DNA, making music something you can hear, see, and touch.

Comment

Inspiration I've been thinking about this for more than ten years: what if you could not just hear a sound, but see it take shape — and keep it?

Back then, the technology wasn't ready. Today it is. Sound Matter Engine turns audio into physical form: upload a track, use your microphone, or play a pure tone — watch frequencies become a live cymatics mandala, issue a unique Matter Certificate, then unlock an AI artwork and a printable 3D object.

Pipeline: audio → cymatics → Matter Certificate → AI visual → 3D model.

What it does Upload audio, Live microphone, or Pure tone lab — free cymatics preview (no API keys). Cymatics engine maps dominant frequency (Hz) to circular Chladni resonant modes — live while playing, or frozen at the playhead when you scrub the waveform and pause. Matter Certificate — one-of-one passport with Matter ID, frequency profile, cymatic seal, and QR verify page. Matter Card — branded PNG snapshot of the live cymatics frame. OpenAI image generation creates a sculpture-style visual from spectral features. Tripo converts the image into a rotatable GLB; download color GLB, STL, or 3D render PNG. How we built it Honest attribution:

Core studio (Web Audio, Pure Tone Lab, frequency-locked cymatics, Tripo/GLB viewer, Firebase + credits) — started in Cursor before Build Week. Build Week extension — Codex + GPT-5.6: Matter Card export, certificate/passport pipeline, judge-ready docs. Runtime AI visuals — OpenAI API. Post–Build Week polish (landing, waveform scrub, verify page, GLB downloads) — Cursor. Author: Beka Januashvili Stack: React, TypeScript, Vite, Web Audio API, Canvas, Three.js, OpenAI API, Tripo AI, Firebase.

Challenges Translating sound into a real cymatics pattern, not generic visualizer art — solved with Hz → resonant mode mapping. Per-moment frequency on uploaded tracks — solved with a spectrogram aligned to the waveform playhead. Reliable judge demos without mic permission — Pure Tone Lab built-in oscillator. Mic silence vs sensitivity (noise gate + canvas freeze). What we learned AI works best here as a creative step in a pipeline, not the whole product. Physics-first cymatics + certificate + 3D tells a clearer story than “another audio visualizer.” A ten-year idea became possible when browser audio, OpenAI images, and Tripo 3D finally aligned. Future vision I imagine Sound Matter Engine for musicians, producers, artists, and anyone curious to see and touch their own voice — a unique digital fingerprint you can verify, collect, and print.

Built with React, TypeScript, Vite, Web Audio API, OpenAI API, GPT-5.6, Codex, Cursor, Tripo AI, Firebase, Three.js

Built With

  • ai
  • audio-signal-processing
  • cloud-computing
  • creative-coding
  • digital-visualisation
  • generative-design
  • machine-learning
  • web-technologies
Share this project:

Updates