We wanted a convenient way to explore different sounds and instruments by ear alone, and efficiently locate an instrument that we like without getting bogged down by technical details.
What it does
The user starts at some instrument, and can branch off into different distinct timbres. They can test out these sounds by playing on a keyboard at the bottom of the app window. Once they select a timbre they like, more different timbres similar to it show up, and the user keeps selecting them until they converge on a single favorite instrument.
How we built it
We used Python, making the graphics display with the PySDL2 library and the audio output with the Pygame library.
The instruments are represented by four dimensional vectors, which are then translated to overtone intensities and envelopes, which constitute the timbre data. This data, along with the pitch form the keyboard, is translated into a waveform to be played by combining simple sine waves and multiplying it by the envelope.
The options for next instrument are generated by taking the 8 alternative corners of the 4 dimensional cube surrounding the current instrument. After each selection, the width of the cube is halved. Once this width is below a certain threshold, the user has converged on an instrument.
Challenges we ran into
It was difficult to translate the few perceptual dimensions of timbre (of which there are about 4) into the infinite dimensional space of actual timbres, in such a way as to capture the most variation of sound. We setting on using one perceptual dimension to encode the attack of the envelope, and three to encode the strengths of three basic overtone patters, corresponding to a full overtone spectrum, an even overtone spectrum, and a high-overtone bump. These were chosen to allow the approximation of piano, violin, clarinet, and flute sounds.
Accomplishments that we're proud of
We're really happy with how our app turned out! We think it looks clean and simple, it has an intuitive design, and it works better than we could have imagined. Although the range of timbres isn't huge, it still covers a wide range of instrument combinations and the possible results are numerous. The way that we convey the different instruments with colors was cool. Typically, more blue coloration in a node means it sounds like a piano or string instrument, green means it sound like woodwind, and red means more bluegrass/feedback type of sounds.
What we learned
We learned more about the technologies we employed, and we learned more about the envelopes and harmonics of different instruments.
What's next for Timbreline
Future work definitely includes changing the space of possible timbres. This is a really difficult task that takes lots of experimentation. We're glad we were able to get out the proof of concept, though. We were also thinking of ways to convey what kind of instrument the user was on visually other than with just colors.