RE:SONANCE
Inspiration
We wanted to rethink how patients experience painful medical procedures. Instead of distracting them from what's happening, RE:SONANCE uses music to transform the sensation itself, allowing patients to stay present and engaged.
What it does
RE:SONANCE transforms a routine blood draw into a four-step musical experience:
- Swab: Touch becomes glowing visuals and delicate sounds.
- Palpation: Pressure becomes the rising pitch of a tuned string.
- Puncture: The needle prick becomes a single bell strike.
- Draw: Blood flow becomes a flowing musical symphony.
The clinician sees the actual procedure, while the patient experiences a celestial harp. Both views remain synchronised, mapping every touch, movement and pressure change to sound and visuals.
Additional features include a simulated heartbeat, breathing guidance, voice assistance, three themes and before-and-after discomfort ratings.
How we built it
RE:SONANCE runs entirely in the browser without installation.
- HTML5 Canvas: Renders both synchronised realities.
- Web Audio API: Generates music and sound effects in real time.
- Physics: Adds natural movement, hand tremors and spring-based animations.
- Web Speech API: Provides optional voice guidance.
- Vibration API: Adds haptic feedback on supported phones.
Challenges we ran into
Sensory mismatch: Turning a physical sensation into music without making the response feel disconnected from the actual procedure was challenging.
Different pain responses: The same touch can feel very different to different people, making a single experience difficult to design for everyone.
Accomplishments we're proud of
- Built a fully playable browser-based prototype.
- Created two synchronised but completely different realities.
- Developed real-time music that responds to touch and pressure.
- Added a session panel for participant testing.
What we learned
We learned that timing is essential to sensory reframing. Even a slight delay can break the experience. Small imperfections make interactions feel more human, while transparency about simulated data builds trust. Most importantly, music can change how people interpret uncomfortable sensations.
What's next for RE:SONANCE
- Real sensors: Integrate heart-rate and electrodermal activity monitoring.
- Backend: Use Django to securely store session data.
- More applications: Extend the concept to dentistry, physiotherapy, paediatric injections and rehabilitation.
Built With
- google-fonts
- html5
- javascript
- web-audio-api
- web-speech-api
