music-to-graph

Inspiration

Our inspiration for creating our project comes from numerous sources. For one, I play an instrument and I thought that doing something with music through programming could be really interesting. We were also fascinated with how apps like Shazam work, and we wanted to try something similar

What it does

There's two sides to the program. On one hand, you can use it to read .wav audio files and convert them into graphs for various uses. The other side allows the user to input a polynomial and listen to what that curve would sound like.

How we built it

We built the .wav file to graph part by using Tarsos-DSP, a java library that uses FFT to predict pitch from the .wav file. From there, it puts the time and frequency into an x and y format, graphing the song using the GRAL java library. It also uses linear regression to neaten up the graph into a more readable graph that's easier to interpret. The graph to music end was written in a similar way, using user input to create a polynomial and obtain its x and y values. From there, the points are put into GRAL to form a graph and the audio is generated using the Tone and STD Audio class from Princeton.

Challenges we ran into

The data taken from Tarsos-DSP was initially very hard to interpret. Tarsos-DSP was a very confusing library, and reading pitch from the .wav file was an enormous hassle Bits of calculus were learned on the fly as we needed to determine slope from graphs GRAL library had an awful scaling function. The initial graph from .wav file was way too messy, and the linear regression to better estimate the music's curves was difficult. The parser to turn text into any sort of polynomial equation proved too difficult, and we had to settle on a simpler implementation where the user enters coefficients of a set polynomial.

Accomplishments that we're proud of

The project's actually finished! It reads .wav files accurately for pitch. The graphs from the .wav files are distinguishable from each other. It actually makes sounds. We have a cool logo. Wrote the interface in under an hour. Had fun!

What we learned

How to use Tarsos-DSP and GRAL library. Sound engineering is super hard. FFT is a very confusing algorithm, and there are also a couple different versions as well. Don't reinvent the wheel; you end up losing a lot of functionality. Don't get ahead of yourself. Take breaks. Get a lot of sleep the night before. Drink protein throughout the night. It really helps you stay awake.

What's next for music-to-graph

Implementing Wolfram API Include multi-threaded implementations Picking up multiple instruments as their own line when detecting pitch Making the graphs more intuitive and interactive.