YouTube videos are fun. But how about making it more involved by participating in discussions while watching? Why not shouting at critical moments and let everyone else watching the same video know your excitement? This is where our app comes in to play!
What it does
While watching a YouTube video, whatever you want to say, just say it out and we will convert it to text that flows through the screen like waves. Everyone else can see your comments on the screen at the same time of the video.
Also, we support smart video search by voice queries. Just ask it what you want to watch in plain English, the search engine will convert it to precise queries and give high quality results.
Finally, you may use our Android app as a remote controller! With it, you can control the pause and play, and send voice comments remotely, all instantly!
Believe it or not, we also support YouTube live videos! Watch the same live stream with people around the world, and share your feelings with people in real time!
How I built it
We built the web app in Node.js. The voice to text conversion is done by Houndify API. Smart search is made possible by Stanford Named Entity Recognizer (NER), a Stanford natural language processing package in Java, followed by query of YouTube search API. Flowing comments are controlled by jQuery scripts. Android phone to web client pairing is done by bridge-connection through the web server by unique session IDs. The comments data are stored in MongoDB. Both the server and MongoDB are served on Linode instances. The domain name is registered on namecheap.
Challenges I ran into
The biggest challenge came from finding the right tools for solutions and learn how to use them under stringent time limits. It took time to figure out what tools are proper for NLP, flowing comments, and Android-web connections. It was also the first time for us to use Houndify API, Linode, Namecheap, YouTube APIs, video controls, etc. We learnt fast and got things done!
Accomplishments that I'm proud of
Bring many pieces of cool technologies together in harmony to build a fun app!
What's next for VoiceWave
Smart video recommendation by machine learning based on user comments data. Add more features to Android app. Add private chat room feature. Improve efficiency, serve on the cloud in the long run.