iPod Synesthesia

Sound wave generation for Believer
Sound wave generation for Anna Sun

Inspiration

The inspiration for this project was a love for the simplicity of retro user interfaces. We tried to capture that old magic and reinvigorate it with the 3d technology of our new age.

What it does

The app takes in any mp3 file song, with its own preset options too-- and delivers an immersive visual experience as it analyzes the tempo, and mood of the track.

How we built it

First, we split up the work into everything done in Unity and other functionality. The python song analysis code and 3D assets were done by Aarushi and Daniel. The Unity magic was done by Tiffany and Lawrence.

Challenges we ran into

One challenge was the integration of the different tools we used in this project. Particularly getting TouchDesigner and Unity to work together was a struggle, as well as Unity with the Python song analysis code. Integrating everything with the virtual reality system in the end without having it crash was a challenge. Some more challenges...

Figuring out an algorithm for color so hues, saturation, and brightness varies
Debugging Touch Designer and designing 3D elements

Accomplishments that we're proud of

One accomplishment was the synthesizing of the song's data through libraries like librosa. We converted the tempo (bpm) and key into numerical values and then used math to turn those into colors for the sound waves to be displayed in.