-
Yoogle
-
-
Inspiration
With the rise of viewbots and more frequent occurrences of clickbaits, genuine video creators - who possibly have the solution of a problem that you're looking for - aren't visible on the Youtube search results. Hence, our platform provides the users with a more logical and sensible way of searching for Youtube videos. We let the contents of the video speak for itself rather than the creator of the video.
What it does
Our platform is a Content-Based Youtube video search engine web application. Based on any search query, our app returns videos which actually have contents that are being searched for. It uses state of the art machine learning and AI algorithms to detect various attributes of the video ( for example, all the texts and objects that come up during the video, all the famous celebrities and landmarks etc.)
How we built it
We have used Flask to make our web application. The front end is based on CSS and HTML. Firstly, we used Python to download each frame of a certain amount of videos from Youtube. Then, we have used Microsoft Azure Cognitive Services (Computer Vision API) to analyze each frame to get details like OCR detected text, objects detected, description of the video etc. Further, we have stored this entire data which follows a well planned schema in Google FireBase realtime database. FireBase realtime database supports real time notifications that allow our search index to constantly be updated with the changes in the database. The search index is made entirely by ourselves which follows an inverted index pattern. Thus, the results from the search index are displayed in our web application.
Challenges we ran into
Our team was new to Azure API's and our project involved a lot of Azure computer vision API calls which were limited to 5000 call with 20 call per minute which limited the amount of data that we could index for our project. In the beginning we struggled a lot in getting desired results for the analysis of each frame of YouTube videos. As we were processing our videos frame by frame (where frame is taken every 3 seconds), this resulted in longer processing time and to parallelize the process we used RabbitMQ with multiple consumers to achieve higher throughput
Accomplishments that we're proud of
Our biggest achievement was to make our project work end to end keeping under consideration the time constraint we had. We were also able to make an application which has a lot of potential in the future use 2 of the biggest API's
What we learned
We all were thrilled to use Azure cognitive API and Firebase realtime database. We learned about decoupled application architectures using RabbitMq to increase throughput. We got a chance to work with the ffmpeg library for video processing. All the feedbacks we got from the industry professionals of Microsoft and Firebase were of great help and surely helped us shape our product in a great way.
What's next for Yoogle
Integration of Azure search for more efficient and scalable data indexing and querying capabilities. Analyzing for videos to provide better search results. In future we can take into account other attributes like OCR on frames to make the video searchable based on the text in the video which would be very helpful to navigate around educational videos. We can also on improving the metric being used for ranking the search results for ex. The relevancy of a search results can be based on tags which are wighted based on the users preferences. Hence providing the user with a personalized experience.
Log in or sign up for Devpost to join the conversation.