Synesthesia

Inspiration

When we initially met in the small room above the auditorium in Salomon Hall, we quickly gravitated towards and found some basic interests that we all shared. Even though our majors and interests all differed, we found some commonalities: music and art. In that tiny crowded room, we began on the rough draft of a virtual reality experience that allowed the user to experience artificial synesthesia.

What it does

We settled on the name synesthesia and our vision consisted initially of an augmented reality experience that could pick up on any of the sounds around you and overlay the programs computer-generated imagery on your device. The sound of waves crashing onto the shore, your favorite song, or even a baby crying would all have a visual representation and the user could figure out what sounds look the best to them. The original vision stayed with us, but the limitation of our "hacking" knowledge called for the dream to change slightly. We settled on using the p5.js program to build a website that portrayed a song and each of our own visual representations.

How we built it

We used Sublime Text 3 and wrote in p5.Javascript to create the shapes and sounds that you see on our websites.

Challenges we ran into

Our initial dream was to create an AR or VR experience, but we lacked the capability to move our HTML links on to a phone. All 5 members of our group were first-time hackers, we had to learn how to understand and use p5.Javascript in a matter of hours and then implement all the new tricks we learned as we went. But this made the process more exciting. As we all explored the p5.js guides and learned how to make loops and change the colors, we were able to make our creations increasingly more complex as we went along.

Accomplishments that we're proud of

Almost all of us had never done anything like this before, so the fact that we learned a new computer language and how to translate our creativity into codes made us immensely proud.

What we learned

p5.javascript, how to add audio to an HTML, how to translate creative visions into lines of code, and how to laugh through frustration.

What's next for Synesthesia

Upgrading the technology so that the visual imagery can match the surrounding sounds of the world and allow people to experience a little artificial synesthesia of their own.