-
The nearly successful website :)
Inspiration
We took inspiration from the concept of sonification, the process of translating data to sound. However, most sonification processes tend to sound a bit ‘screechy’ so we wanted to find a way to produce a melodic sound.
What it does
Welcome to the Sounds of the Universe, a program that transforms your favorite images into beautiful songs. Unlike regular sonification, this program outputs musical notes that sound smooth and snazzy. Simply upload a jpeg to the user interface and be ready to hear a sound unique to this image.
The process is as follows: When an image is uploaded, the program slices it and returns the median of RGB (red, green, blue) values from the pixels of each row. Then, the RGB values are converted to HSV which stands for hue, saturation, and value. From the information given by the HSV of each row of the image, the program is able to produce a sound with varying melodies. The hue indicates the note to be played from a piano note frequency dictionary, the saturation translates to the volume of the note played and the value sets a duration for the length of time that said note is played. In the end, a melody is played from the image input!
The goal of this whole project was to produce a sound that is pleasant to the ear and unique to each image all while dealing with the physics of sound waves and frequencies to transform color to sound.
How we built it
The program was made in collaboration with Python 3. Notably, imports such as scipy.io.wavfile and pygame to produce the actual sound from given frequencies. One of our teammates attempted to create a user interface using both HTML and Javascript. She was very close to being successful however, we were unable to get the Java program to read the Python script.
Challenges we ran into
• As indicated above, the user interface was almost a success, but the JavaScript program was unable to read the Python code which contained the core of our project. Hopefully, with more time in the future, this problem could be an easy fix. • Generally, dealing with several different languages at once while getting them to interact was a big challenge. • In the beginning, we wanted the musical notes to sound like an actual piano however, we realized that with the code we wrote, we would have to deal with Fourier transformations and superposition how multiple waves with different frequencies. We fell short of time to do so. • We also wanted to be able to download the sound as an mp3 file but the object type was not convertible to mp3.
Accomplishments that we're proud of
We are extremely proud of the fact that we were able to take any image and make it into sound in the span of a 24-hour deadline. Notably, we created a nearly functional website in HTML and Java and programmed noise generation in Python.
What we learned
While accomplishing this project, we learned a lot about sound waves and different methods of data representation and their various uses in scientific communication. Our knowledge was expanded in the world of programming and our problem-solving skills were improved.
What's next for Sounds of The Universe
We would like to make the program more interactive by allowing the user to change some parameters to their liking and allowing them to get a downloadable mp3 of the sound. It would also be of interest to make sound into images.
Log in or sign up for Devpost to join the conversation.