Inspiration
Halloween costumes change how you look, but not how you sound. Voice changers usually pitch-shift a recording, so every monster ends up sounding like a chipmunk or a slowed-down tape. We wanted the voice itself to wear the costume: a creature whose throat, lungs and vocal folds are simulated, and whose face moves with every sound it makes. Costume Contest asks for a haunting interface that serves the app, so the creature on the pedestal is both the costume and the main readout.
What it does
Creature Voice Studio is an offline monster-voice lab for Android.
- Creature DNA Mixer. Five strands (Monster, Alien, Robot, Human, Animal) always add up to 100%. The blend drives the vocal folds and lungs of a physical voice model, a Rust effect rack (formant and pitch shift, sub-octave, growl, ring modulator, vocoder, metallic comb, flanger, bit-crusher, reverb and more), and splices in a real animal airway (songbird, seal, alligator or elephant) that breathes with your specimen's words.
- A Night Lab interface. Near-black glass, ultraviolet and toxic-green light. The colours carry meaning: toxic green marks what is live (the pitch/jaw/nasal readouts, active tabs and chips, Splice, playback progress, the glowing incubator while a voice renders) and ultraviolet marks what you choose (the selected specimen, primary actions). DNA sliders glow in their strand's colour, spores drift behind the stage, and the specimen stands on an obsidian plinth whose ring and floor glow pulse with its voice.
- 3D specimens. Eight sculpted heads (ogre, demon, alien, robot, beast, skull, pumpkin, patchwork) are rendered with PBR materials. The jaw, lips and tongue follow the engine's articulator telemetry, the eyes blink and follow your finger, and the pedestal ring glows with the level. Horns, antennae, a visor or a collar grow and the hide recolours as you splice. Each preset (Swamp Ogre, Demon Lord, Zeta Reticulan, Gamma Robot, Werebeast, Pumpkin King, Cyber Lich, Banshee and more) wears its own head.
- Text-to-Voice in English, Indonesian, Spanish, raw IPA and Creature Tongue, a deterministic alien language with three dialects.
- Studio: a Singing Studio, a Beatbox Maker (live pads and a 16-step sequencer) and a Sound FX Generator (18 synth categories, 10 physical creature vocals, animal calls).
- Voice Morphing between two characters mid-sentence, or morph your own microphone recording.
- A clip editor, a project vault, and WAV/MP3 export to Documents and the share sheet.
There are no samples in the app: every sound is synthesised on the phone. No INTERNET permission, no account, no ads, no paid APIs.
How we built it
- Engine: the CHIMERA physical voice engine (MIT, by Jameson Hodge) is vendored unchanged in Rust. Our own
creature-labcrate adds the DNA model and effect rack, Indonesian and Spanish G2P, Creature Tongue, the SFX synthesiser, clip editing, WAV encoding and an offline renderer that records 60 fps telemetry frames. Both compile into onecreature.wasmwith no imports. - Three threads, one module: the wasm is compiled once and instantiated on the UI thread (planning), in a render worker (offline renders, WAV/MP3) and in an AudioWorklet at 48 kHz (live Incubate pad, beatbox, mic monitor). Long renders stream to the speaker before they finish.
- 3D: the heads are generated, not hand-modelled: Python scripts drive Blender 4.5 headless, sculpt ~500k-face heads with skin detail, and bake normal and detail maps onto ~15k-triangle game meshes (2.1-2.5 MB per GLB). three.js renders them in the app.
- App: React 19, zustand, three.js and Vite, packaged with Capacitor 8 for Android (minSdk 24, targetSdk 36).
- Tests: 13 Rust tests in
creature-lab(rack finiteness and level from 16 to 96 kHz, DNA mapping, G2P tables, SFX determinism, WAV round trip, editor ops, every render job, the animal splice), a wasm export and benchmark check, TypeScript typecheck and a licence gate.
How Kiro was used
We built it with Kiro CLI. Everything it used is committed in .kiro/ (not in .gitignore). Details: docs/KIRO.md.
- Spec-driven development.
.kiro/specs/creature-voice-studio/has EARS requirements (R1-R11), a design with the thread architecture and the DNA mapping as formulas, and a 7-phase task list where every task cites its requirements. Tasks were ticked only when verified, and each session is logged indocs/DEVLOG.md. - Steering. Six files.
product.md,tech.mdandstructure.mdare always included (the product promise, "never placeholders", offline-only, no allocation on the audio thread, keep upstream crates byte-identical).audio-dsp.md,specimen-3d.mdandlicensing.mdload only for the files they govern (crates/creature-lab/**and audio code;src/three/**, Blender scripts and models; manifests and notices). - Agent hooks. Six hooks: Rust tests when lab code is saved, a typecheck when TypeScript is saved, a headless Blender rebuild of the heads when the modelling scripts change, a licence gate on dependency changes, a pre-tool hook that blocks staging the signing keystore, and an end-of-turn reminder to tick the tasks and write the devlog.
- MCP. We wrote
creature-adb, an MCP server that lets Kiro drive the phone over ADB and the WebView over DevTools: install, launch, screenshot, run JavaScript in the app, read console logs, switch screens, tap, benchmark a render and read memory/CPU/frame stats. Testing on the real Snapdragon 680 phone this way (MCP tools and ADB) is how the causes of choppy playback (the idle live engine starving the audio thread) and of silence (a suspended AudioContext) were found, and the fixes were checked with real taps. - The spec followed the product. The first design showed an anatomy hologram of the airway. After testing on the phone it was replaced by sculpted heads, and that decision went into steering ("no anatomy diagrams: the specimen is the show").
- Kiro CLI was the Kiro app used for the whole build, working from the spec, steering, hooks and MCP config committed in
.kiro/. No custom agents or Kiro powers were used.
Challenges we ran into
- A physical airway is expensive. On the Snapdragon 680 test phone offline renders run at about 3x real time, so speech is rendered in a worker and streamed, and the 3D view drops to one frame per second while a clip incubates.
- Choppy playback on the phone: the live engine was running in silence on the audio thread and starving clip playback. It now sleeps when idle.
- A clip rendered but nothing played: the AudioContext had started without a touch and stayed suspended. It is now resumed on every touch, on visibility and before every playback.
- Making laughter sound like laughter: each "ha" is now an expiratory pulse with a falling pitch, and the echo and reverb are capped so they do not fill the gaps.
- Turning primitive shapes into convincing heads inside a mobile budget: voxel-fused sculpts, fillets, and baked normal and detail maps on ~15k triangles.
Accomplishments that we're proud of
- Every sound is synthesised from a simulated throat, with real animal airways spliced in. No samples ship.
- A creature that lip-syncs to its own physics, on a mid-range phone, offline.
- Text-to-voice in five "languages", including one we invented.
- A Kiro setup where hooks and our own MCP server tested on the real device, not just on the desktop.
What we learned
Steering that is scoped to file patterns kept the DSP, 3D and licensing rules precise without bloating every request. Measuring on the real phone mattered: both audio bugs were invisible on the desktop and obvious once Kiro could tap the screen and read the WebView. And a "costume" works best when it does a job: the head is there because it shows the jaw, lips and tongue the voice is using.
What's next for Creature Voice Studio
Shorter render waits on low-end phones, more heads and presets, and a Google Play release of the signed AAB. No accounts, no cloud. Known limits today: on a Snapdragon 680 phone a render takes several times its length (a 4.5 s laugh took about 25 s in our last measurement), so long clips wait in "Incubating" first. Some heads still need polish (the pumpkin's eyes read as yellow balls rather than candle light).
