1 in 8 Americans & 360 million people globally live in partial or complete silence due to hearing loss. Cultural events (performing arts & social gatherings) are a powerful way to bring communities together; however the Deaf & Hard-of-Hearing (DHH) are often excluded their benefits, as such events often rely on sound to deliver their experience or message. Venues & artists often lack the resources or are not aware that their events or mediums are not accessible to those who cannot hear. There is a need to translate sound into other senses & make cultural events more accessible.
A Deaf musician defined the needs and functionality of the Cymatic Lighting system. It was important that the lighting visualization gave meaningful cues as to the nature of the sound. Stereoscopic features of sound were translated into spatial movement to establish a relationship between where sound was panned in space and the visual pattern is displayed. Qualities of sound such as frequency and amplitude are mapped to similar qualities of light luminosity and hues via algorithms. It is important to the end users that detailed information about sound is accurately translated into visuals so that they can be made aware of audio and music cues.
Our approach takes a scientific & technical approach to exploring the relationship between sound and light by incorporating Cymatic principles (the study of sound made visible) which helped us improve the concept of visual sound beyond simple blinking lights from prior technology. Earlier versions of our solution were reported to be not as clear at visually depicting the difference between individual notes being played on an instruction so we incorporated additional code and algorithms to respond to the acoustic envelope (dynamics) of audio which users felt was much more responsive to nuances of musical performance.
More advanced tests of our solution that incorporated multiple strips of LEDs in a 2D grid even allowed our users to see subtle differences in timbre e.g. it was possible to see the difference in harmonics and distribution of frequencies that lead to recognizing the difference between a percussion or string or synthesizer instrument.
Deaf users reported that incorporating the Cymatic Lighting system into the environment (architecture or furniture) allowed for them to enjoy a more engaging experience in the context of a live or recorded music performance. They were able follow the rhythm and pick out individual instruments such as guitar or drums from a mix and reported that this experience of music was superior to anything they had tried before.
When exploring possibilities for additional applications, there was interest from Deaf users in installing the technology as a safety / alert system, since it can easily convey any audio signal into an accurate expression of light and movement. This means it would be able to indicate the difference between a doorbell or fire alarm alert if installed as part of a smart home lighting system for the Deaf & Hard-of-Hearing.
In the future, with further development and funding, we aim to incorporate wireless control of the system from mobile devices for such uses as streaming wireless music via bluetooth and integrating a 2-way alert feedback so that users can receive notifications and even have similar light display visualization displayed on their mobile device screens, further extending the concept demonstrated by the hardware solution.
Many users repeatedly expressed that they would like to have the technology at home for their own enjoyment or wished that performing art venues would incorporate similar approaches to technologies so that their events would be more engaging and accessible to the Deaf community.
If more venues were made aware of the needs of the Deaf and incorporated our Cymatic Lighting concepts, our users felt it would open up more opportunities for them and others in the Deaf community to attend social & art events and not be as isolated from the hearing world.
Built With
- arduino
- neopixel-library
- neopixels
- sparkfun-msgeq7-shield
Log in or sign up for Devpost to join the conversation.