Synaesthesia is a neurological phenomenon in which perception of a certain stimulus, such as the musical note ‘A’, involuntarily elicits another seemingly unrelated sensation, such as seeing the color red. One of our group members, Misha Ali has synaesthesia and wanted others to know what this experience feels like.
What it does
We built a prototype that stimulates one of the most common type of synaesthesia, the color/sound association. It reacts to straight notes and/or songs by flashing LEDs in different colors. The lights move in a circular motion and its position is dependent on the frequency fed into the mic and the intensity of the colors changes based on the change of amplitude.
How I built it
We build the prototye using teensyduino, and LEDs, and coded using MATLab and Arduino.
Challenges We ran into
We were having difficulty cancelling out the noise of the environment and our initial model was picking up these sounds and was reacting to them. The colors that the LEDs were emitting were too bright and this intensity was initially not controlled.
Accomplishments that we proud of
We fixed the problems that we faced - we cancelled enough noise from the environment that when music is played MUSyc visibly reacts to the song. We also controlled the movement of the LEDs depending on the note being played - making it a useful tuning device. And we programmed the prototype in a way the intensity of the light changes according to the difference in amplitude - creating a way to experience dynamics visually.
What we learned
The members of this team come from different backgrounds and each had something to give and something to learn. This project heavily relied on Electrical engineering concepts and we had a graduate EE student in our midst who took the time to teach technical concepts so everyone could participate in the discussion and contribute.
What's next for MUSYC - Music-Synaesthesia-Color
We would like to see this product put to three main uses - in music schools, in performance, and as a hand held device.
- This is a pedagogical tool for those beginning their training in music. It's a visual aid to identify and differentiate between similar notes. If you can associate a color to a note, this can increase how students remember pitch.
- There are two different types of performances this product can enhance - a music hall with a backdrop to give a visual aid to the music being played (ex: orchestra, opera) and an interactive performance that multiple people can take part in (raves with people's clothes reacting to music).
- We think this product can be very useful as handheld device as well. It can be a tool for the deaf and hard of hearing to take with them wherever they want to as this can include them in an experience they would otherwise be missing.