Inspiration

We wanted to make visually appealing and interactive, for example, displaying the lyrics of a song in a unique way in order to make it more captivating. (And we just wanted to make something cool)

What it does

It has a sea of letters, and once the computer/phone hears a voice, from the sea of letters words rise up. These words happen to be what the voice is saying. The sea has properties of a sea. This includes waves that increase with volume. Yes, very sea-like. In addition to all this, it has a 360 view. You can move your phone/computer in order to look around you. It works on the google cardboard as well, making it VR friendly. This runs completely in a browser, ensuring maximum cross device compatibility.

How I built it

In all actuality, we used javascript with a three.js module. We used this to make the sea of letters, using a lot of math in order to figure out what the velocities of the letters should be and even more math in order to work with the gyroscope. We then used a speech to text module, did a bunch of math to find where the letters were and did even more math to figure out how to get it in front of the person no matter what way they were facing, and stay in front of the person even if they moved around. In order to do the cross device compatibility, we had to make sure that we accounted for the fact that if our website was pulled up on a phone, it would make fewer letters and not move as fast in order to account for the processing speed differences. We used a three.js extension for VR with the Google Cardboard. The modules chosen and some of the design was influenced by a project called created-index.

Challenges I ran into

As mentioned above, in order to make it so that it flows smoothly without lagging on every device, we had to make some changes for the phone that we did not expect to have to do at the beginning. We also ran into countless bugs during the development process, most of which we were able to squash.

Accomplishments that I'm proud of

The fact that none of us knew how to javascript or three.js and managed to figure it out. In Roshan's case, the fact that he was able to apply derivatives in order to make the wave. We were able to diagnose problems as we went along and though some were difficult to fix (getting the text to face the user continuously), we were able to get through them. The fact that we made it so accessible is also something to be proud of.

What I learned

We all learned to use JavaScript for the first time and used three.js as well, despite none of us having much 3D graphics experience. We also learned that calculus is actually used in real life.

What's next for SoundIt!

Important links

Regular version: http://tkadur.github.io

VR version: http://lucychenrobotics.github.io

Source code: https://github.com/tkadur/YHack2016

Built With

Share this project:
×

Updates