Inspiration
Body Resonator, also titled Corpus Resonans began with a question: what if an anatomical atlas could be played directly like a musical instrument?
The main interaction reference was Pink Trombone. Its most compelling idea is that participants do not operate a synthesizer through an abstract control panel. They touch visible anatomy, and continuous physical manipulation creates an immediate change in sound. I wanted to expand that principle beyond the vocal tract and explore the whole body as a speculative acoustic system.
The visual inspiration came from nineteenth-century anatomical engravings: aged ivory paper, intricate crosshatching, muted arterial reds, blue-green veins, numbered figures, delicate leader lines, and marginal scientific studies. The goal was to make an old medical plate appear quietly alive—beautiful, tactile, and slightly uncanny.
The project was created at Constellation Labs, during a three-day hacklab at Espronceda institute of art and culture . Because of the short timeframe, I focused on the thorax instead of attempting a complete playable body. This gave me a connected group of organs with contrasting musical behaviours: breathing lungs, a beating heart, resonant ribs, a gurgling stomach, and a playable esophagus.
The biological references inform the interaction metaphors, but the instrument does not claim that organs possess single scientifically established musical frequencies. Its tunings, resonances, and cross-organ relationships are artistic constructions.
What it does
Body Resonator is a publicly accessible web instrument in which the anatomy itself is the interface. After the participant clicks “Touch to awaken the plate,” the body begins breathing and beating automatically. These autonomous processes give the impression of encountering a living illustration rather than opening a conventional music application.
The lungs expand and contract together in a repeating respiratory cycle. Participants can grab either lung to interrupt its automatic motion and shape the breath manually. Stretching creates inhalation, compression creates exhalation, and faster movement produces stronger airflow. Moving a lung vertically changes its resonant pitch, and that pitch remains after release. Breath Texture moves between airy, focused, and raspy sound, while Effort changes articulation from gentle to forceful without altering breathing speed.
The heart continues beating at an adjustable rate. Gripping and deforming it adds wet friction, sticky movement, and subtle liquid separation sounds without interrupting the automatic pulse. A circular circulation study can also change the heart’s apparent size and sonic depth.
Individual ribs can be struck or scraped. The contacted rib visibly responds, while position and gesture strength influence a short percussive resonance. A bone-thickness study changes the depth of the rib sound. The front of the rib cage can also be opened, exposing the softer organs and making them easier to play.
The stomach supports squeezing, kneading, and circular stirring. Slow gestures create drawn-out internal movement, while faster gestures produce denser bubbling. Activity controls the frequency and clustering of spontaneous digestive sounds. Viscosity changes their source character from watery movement to thick, sticky churning.
The esophagus uses real throat and gargling recordings. Dragging along the tube moves an implied bubble through it, while sideways pressure constricts and roughens the sound. Its Character control blends between natural wet bubbling and exaggerated rubbery, almost vocal gargles.
A gesture-loop recorder captures organ movements rather than recording the final audio. A performer can record a bodily phrase, loop it, and continue playing live. Because the gestures are replayed through the instrument, resonance and effects remain adjustable during playback.
How I built it
I began by defining a strict MVP: one visually convincing lung with a continuous, satisfying sound. Before refining the illustration, I built a crude interaction prototype that translated pointer pressure, direction, position, and velocity into normalized control values.
The application keeps input, organ state, rendering, and sound generation separate. Mouse, touch, stylus, live performance, and recorded gestures all produce the same organ-control data. The visual renderer and audio engine consume that shared state independently. This also creates a path toward future VR controller or hand-tracking input without rewriting the sound models.
The instrument is built with TypeScript, React, Vite-compatible tooling, SVG, CSS, and the Web Audio API. The audio architecture was also designed around RNBO-style parameter boundaries so synthesis ideas developed in Max can be incorporated later.
SVG was essential to the visual system. Each organ has its own artwork layer, transformation anchor, hit region, annotations, and active feedback. The organs use restrained scaling, rotation, and segmented deformation instead of full physics simulation. This kept the interface responsive while preserving the engraved illustration style.
Public-domain anatomical sources informed the artwork. The primary historical reference was the 1886 “Blutgefässe des Menschen” plate from Meyers Konversations-Lexikon. Gray’s Anatomy material supplemented the thoracic structure. The final composition is a new landscape plate rather than a direct reproduction. Asset sources and transformations are documented separately.
The audio engine combines procedural synthesis, samples, and physical-model-inspired resonators. Lungs use coloured noise, envelope shaping, human breath texture, irregular modulation, and tuned resonant filters. The heart combines an independent paired pulse with sample-based tissue deformation. Ribs use short noise impulses and friction excitation. The stomach combines recorded gurgles with procedural bubbles and filtering. The esophagus crossfades and processes two source recordings without restarting them on every pointer movement.
Each organ has separate dry and resonant paths. A global Resonance control blends the untreated bodily sound toward individually tuned resonators. Delay and reverb are parallel wet-only returns, followed by a master volume, high-pass filtering, compression, soft clipping, and final level protection.
Automatic heartbeat, breathing, and digestion use the audio clock rather than animation frames. Animation reads the same cycle state, keeping sound and motion synchronized. The loop recorder also schedules repetitions against an absolute audio-time origin to avoid gradually drifting out of time.
Development proceeded incrementally: first one lung, then paired breathing, heart, ribs, global resonance, effects, stomach, the gesture recorder, and finally the esophagus. Each major sound was auditioned in isolation before being integrated into the body.
Challenges I ran into
The hardest challenge was balancing anatomical detail with playability. Accurate-looking lungs behind a dense rib cage were difficult to grab. I widened the anatomy, adjusted organ hit regions, and added a rib-opening mode. The visible ribs and their interaction masks also had to remain aligned throughout the complete animation.
Audio resonance required several revisions. An early version sounded like an unrelated high-pitched tone. Increasing feedback then produced unstable buildup that could overwhelm or crash the audio. I replaced this with bounded, damped resonators, smoothed parameter changes, level-aware dry/wet curves, and final dynamic protection. The heart grip initially sounded like a low kick rather than wet deformation. The stomach had a similar problem with noisy percussive transients. Both improved when I stopped trying to force procedural synthesis to imitate every detail and introduced carefully shaped sample material.
The esophagus was especially demanding. Early versions were misplaced, too straight, visually sliced off, or hidden behind the closed ribs. I isolated the source artwork on transparency and discovered that some defects belonged to the asset itself rather than CSS. The final version uses repaired artwork, intentional occlusion behind the heart, and visible cutaway sections whose hit regions match the rendered tissue.
As features accumulated, the controls and historical annotations began overlapping. The solution was a structural layout pass with dedicated header, footer, central stage, and left and right control columns—not another series of individual coordinate adjustments.
I also encountered a preview-specific audio bug: hot reload preserved the interface’s “awake” state while destroying the previous AudioContext. The page looked active but was silent. Audio sessions now reset visibly after a development refresh and require a fresh user gesture.
Accomplishments that I'm proud of
I am proud that the finished piece feels like an instrument rather than an illustrated collection of sound triggers. Every major organ supports continuous expression, and the same gesture produces coordinated visual and sonic change. The autonomous breathing, heartbeat, and digestion make the plate feel alive before anyone performs. At the same time, participants can interrupt those processes and take direct control without abrupt jumps.
The gesture recorder is another important achievement. Recording normalized performance data instead of stereo audio means a loop remains responsive to later changes in resonance, texture, viscosity, and effects.
What I learned
The project reinforced that interaction design and sound design cannot be developed separately. A sound may be convincing in isolation but fail when its gesture feels wrong, and a beautiful animation can feel disconnected if it does not share the audio’s control state.
I also learned to prototype the riskiest sensory relationship first. One expressive lung revealed more about the project than building an entire static body would have.
Finally, hybrid audio was consistently more effective than ideological purity. Procedural synthesis provided responsiveness and variation, while recordings supplied organic detail that would have been expensive to recreate convincingly.
What's next for Body Resonator
The next step is a second anatomical plate exploring abdominal and nervous-system interactions, including winding intestinal resonators and impulses that propagate through branching nerves.
I would also like to add a restrained performance mode with longer gesture loops, overdubbing, Web MIDI integration, and multichannel spatial routing for installations.
The VR version would preserve the plate rather than replacing it with a fully modelled body. My preferred approach is a life-sized anatomical page mounted vertically in space, with organs separated by shallow layers. Participants could lift a lung or heart slightly away from the paper while continuing to manipulate its sound.
Most importantly, future development will keep the project’s defining principle intact: the body should remain the instrument, not decoration placed around a conventional synthesizer.
Built With
- audiocontext
- cloudflare-workers
- css3
- git.
- html5
- openai-sites
- pointer-events
- react
- rnbo-ready
- svg
- typescript
- vinext
- vite
- wav-sample-processing
- web-audio-api
- web-workers/audio-scheduling
