KANKAKU Frame

What it is

KANKAKU Frame is a browser-based visual instrument. It listens to a microphone, a local music file, or its own generated pulse, then turns that sound into the breathing, colour, surface, and silhouette of a living 3D organism.

Five organisms share the same sound but respond as different bodies. Everything happens locally in the browser; microphone and music data are never uploaded.

Why I built it

Most music visualizers either need setup, ask for an upload, or feel like utilities. I wanted something immediate enough for a visitor to understand in seconds, but expressive enough to leave running during a performance.

What is new for Build Week

An earlier KANKAKU visual study existed before the hackathon. During the submission period I rebuilt it as a standalone v3 and meaningfully extended it:

• five distinct ray-marched organisms;
• microphone and local-file analysis;
• Demo Pulse, a locally generated 120 BPM pattern that needs no permission or media file;
• live beat and energy telemetry;
• beat-onset detection;
• Auto Director choreography that moves through the organisms in time with the sound;
• responsive controls, reduced-motion support, testing instructions, and automated checks.

The repository includes a dated Build Week change record and identifies the pre–Auto Director restore point.

How I used Codex and GPT-5.6

I used Codex with GPT-5.6 as a hands-on build partner: inspecting and reconciling prototypes, designing the shader and audio-analysis pipeline, reasoning through Web Audio lifecycle issues, translating the judging requirements into a concrete checklist, and testing the finished experience in desktop and narrow mobile layouts.

I kept the central product and visual decisions: a private local-audio workflow, five deliberately strange organisms, one compact control surface, and an immediate generated demo for judges.

How to try it

Open the web app and press Demo Pulse. Auto Director starts automatically, shows a 120 BPM readout, and changes the organism every four beats. No microphone permission is required.

For personal input, use Start Ear or load a local track with Music.

Built With

• codex
• glsl
• gpt-5.6
• javascript
• three.js
• web-audio-api
• webgl