MEmento

MEmento music box
MEmento sensors
MEmento process
Memento group

Inspiration

We are surrounded by data and systems designed to optimize us, but fewer tools help us slow down, reflect, and connect with ourselves and with others. We asked: what if your body’s signal data could become a ritual, not a dashboard?

MEmento is our answer: a tangible object that turns physiological traces into something you can listen to: “hear what you carry”.

What it does

MEmento is a hand-cranked music box that transforms physiological signals linked to emotional arousal into a short musical composition.

Input: two biosignals — galvanic skin response (GSR) and heart rate (HR)
Output: a ~30s audio piece generated as MIDI and synthesized into sound
Interaction: turn the handle to play; stop turning to pause

How we built it

Biosensing + preprocessing
- We read GSR (skin conductance) and HR, then cleaned and normalized both to a 0–1 range so they’re comparable.
- We extracted features with a sliding window (20 samples), computing: mean, standard deviation, slope, and peaks — producing a sequence of 4D feature vectors over time.
Learning temporal patterns (LSTM encoder–decoder)
- We trained two LSTM encoder–decoder models (one for GSR, one for HR).
- Each encoder compresses the feature sequence into a latent vector capturing time-series dynamics (e.g., stability vs variability), and the decoder reconstructs signals to learn meaningful representations via reconstruction error.
- We use these latent values as musical control signals (GSR shaping melody, HR shaping tempo/rhythm).
Music generation (MIDI → audio)
- The generator automatically selects two instruments from the full MIDI set (0–127) based on combined physiological state.
- It builds a ~30-second timeline (~128 notes) in phrases; pitch steps stay within a C-major scale for smoothness, while physiology modulates pitch/velocity/timing.
- HR influences note duration and rhythmic density; we write notes to MIDI tracks, then synthesize to audio.

Challenges we ran into

Making “physiology → music” feel meaningful (reflective and listenable), not like a noisy sonification.
Cleaning, normalizing, and syncing signals so small bodily changes can shape music without breaking it.
Translating learned latent features into musical parameters with enough structure to sound coherent, but enough variability to feel alive.
Building hardware + ML + music generation end-to-end under hackathon constraints.

Accomplishments that we're proud of

A full pipeline: biosignals → features → LSTM latent representations → MIDI composition → audio output.
A simple, legible ritual interaction: crank-to-play / stop-to-pause
A concept and narrative that reframes personal data as reflection and connection — not productivity or optimization.

What we learned

The “translation layer” is everything: preprocessing, feature design, and musical mapping determine whether the output feels like you or just “data”.
There are multiple ways to translate bodily signals into musical signals. Deeper research into musical frameworks is needed to produce consistent outputs.
LSTMs shine when you treat physiology as a story over time, not independent moments.

What's next for MEmento

Connect to data via wearables.
Refine the composition engine: richer harmony/scales, clearer phrasing, and stronger mappings between latent features and musical structure → Try Terry Riley framework.
Refine the physical form: explore different shapes and improve embodied interactions
Explore sharing: exchanging “musical traces” with others or creating multiple compositions