The overall immersive experience
The searched for keyword highlighted
Our extension's logo
We are always on the lookout for a faster and smoother experience, then why should we waste time scavenging through videos on a particular topic only to hear a few important lines, or manually scout to find that one fact/joke that we once heard. We feel that this is wrong and so we came up with audSearch - a video parsing tool which selectively highlights certain searched for segments of videos and gives general viewer trend on the topic.
What it does
audSearch gives the user the ability to choose a sequence of words which would then be parsed in the video. The user can choose any of the sections and can selectively hear about the topics of interest rather than going through the entire video. But wait there's more, we felt that likes/dislikes and views are not always sufficient explainers of a video, so we came up with a functionality of color coded entity level sentimental analysis. This would allow the users to see what past viewers have felt like while watching the video.
How we built it
We worked on creating the tool as a google extension to corroborate the existing youtube functionalities. Having coming up with the general control over the features and design of the extension, we worked on using Houndify's API to do voice to text conversion. Such a conversion allows us to search the video and highlight specific keyword occurrences in the videos. Afterwards, we spent our efforts coming up with providing our users a greater experience by providing past viewers sentiment analysis on the topic. This allows them to learn more about the video and the way people are thinking about it, as compared to mere likes/dislikes. All along the process, we kept on providing small surprises such as the pop up window, and the amazingly creative spiral loading bar.
Challenges we ran into
There were a couple of challenges we ran into. Firstly working with the API, getting the video files into recognizable format took a great efforts and challenging tricks from us. For the algorithm to work the way we wanted it to, the file had to broken down into small chunks, which had to be learnt. Creating a google extension came in quick, however designing it to communicate with youtube and the backend made us think/experiment a lot. Since everyone was assigned a specific sub-task, towards the end meticulous effort was also required to merge everything into the app we had imagined.
Accomplishments that we're proud of
We are really proud of finally making what we thought of. The first time we had thought of this idea, we weren't even sure if it was possible but seeing so many people working and taking challenges head first, we gave it our best shot. Making the front end communicate smoothly with the back end and seeing the first results of using the Houndify API got us super excited :)
What we learned
These 36 hours were an amazing learning experience. Each of us learnt something new which we had never done before - be it back-end development, working with APIs or google extensions. We took on challenges and enjoyed every bit of the competition.
What's next for audSearch
Although we are proud of what we came up with in these 36 hours, we plan to make a couple of improvements to audSearch. We are thinking of extending features to work on non-youtube videos, adding a functionality to attaching Postmates API to be able to place orders while watching your favorite recipe videos.