Inspiration
A cappella means “singing without instrument”. You may not know this, but human voice is the most amazing musical instrument due to its ability to produce unique sounds that are delicate and flexible. Unlike any other instruments, human voice is capable of the instant transmission of emotions into sounds, which is probably why we all love listening to musics, and more importantly, have the intention to sing aloud. However, many of us apprehend singing, because we’re afraid that the beauty of music could be destroyed by our off-key voices.
This puts stop to an essential way of expression that is potentially damaging to our mental health. Our team wants to find a way that lets people to appreciate their voices and enjoy the musical potential in them.
A cappella is the key. This form of singing breaks down music to its simplest root, and it uses different sound track that sings simultaneously to add layers and depths to the quality of music. When songs are made into A cappella, the self-expressive aspect of musics is emphasized. Research shows that A cappella benefits people emotionally by brining spiritual uplift, mood enhancement, and confidence.
What it does
Our team built a VR game where A cappella harmony is generated after the players sing. We intend to create an immersive A cappella experience for every player that helps them enjoy the benefits of singing music and appreciate their singing. We want to make singing an accessible curative experience for everyone, regardless whether they can “sing or no”
How we built it
We used Unity to create our VR game environment, and we used Python database Pydub to process the shifts in pitch. We then connected Python to Unity using a server that we built, which allows us to record the sounds in Unity but process the sounds with Python.
Challenges we ran into
When lowering or elevating the keys, the wavelengths of the sound wave changes as the amplitudes of the sound wave changes, which means the lower keys have slower speeds while the higher keys have higher speeds. To make the sound waves overlap, we broke the waves into segments and write an algorithm that match the waves segments by segments. Then, we smooth out all the conjunctions to make the music sounds just as normal.
We wrote the code for shifting the pitch of the sounds using Python, because we needed to use Pydub. However, Unity does not support Python codes, but we need Unity to record the sound so then we can shift the pitch of it. In order to do this, we built a server that allows the sound recorded in Unity to be transmitted to the Python code as .wav files. After the codes process the sound, we send the new sound tracks back to Unity.
Accomplishments that we're proud of
We came up with an amazing idea that all three of us believe have great potentials. As university freshman who just started learning computer science, we were able to use the tools that we have to generate as much as possible. The final product might not be perfect, but we put in a lot of effort to make it happen, including learning lots of developer tools from scratch and tirelessly finding solutions to the numerous obstacles we encountered.
What we learned
We learned how to use developer tools like Blender and Unity. We learned how to shift pitch using Pydub database, We learned how to connect Python codes to Unity.
What's next for PITCH PERFECT
Right now we are not able to implement real time shifts in pitch for the sound track, which diminishes user experience a lot. In the future, we hope we can implement real time shifts to give the users more immersive experience of singing A cappella.
Log in or sign up for Devpost to join the conversation.