Course Concepts Detector using Google APIs

Inspiration

Have you ever missed a class or felt so lost in a tediously long lecture that you feel like you don't have time to learn the material? Our Course Concepts Detector using Google's speech-to-text API helps solve just that problem!

What it does

Our project takes in a video file of a lecture or any other video, parses the video and separates it into the major concepts that are covered. From there, it prioritizes the words based on frequency of use in the video vs. frequency of use in US-en and displays the major concepts along with their time stamps, so that they are easily found in the video.

How we built it

We wrote our code using python to parse the text that the Google API outputted. Once the text was parsed, we created sets and lists and used Parse_Timestamps to get rid of the fluff words. From there we used an average frequency function to display the most commonly used nouns/concepts. Since we used a time stamp format, we also had access to the specific time these words were displayed. With this information, the user can easily find the most important information from the video.

Challenges we ran into

1. 60 second time limit for converting video to text
2. Unfamiliarity with google APIs and how to integrate it with our program
3. Accuracy of the speech-to-text converter

Accomplishments that we're proud of

We're very proud that we now have a functional program that we can use to split videos into the major concepts. This project allows us to see, tangibly, the power of coding and using built-in libraries such as Google's APIs. We can all personally use this software to save us time with our coursework.

What we learned

Firstly, we learned to function effectively as a team and seek help from mentors/experts to help us with our questions. This is an indispensable lesson that will help us in any team scenario, regardless of the specific application. Secondly, we learned how to break down a large problem into smaller, manageable problems that can be designated to specific individuals, so that it can be attacked efficiently as a team. Thirdly, we fostered our creativity and confidence with coding by thinking of a problem that is both useful and doable, even on a short time notice. Finally, we learned how to combine software at multiple layers in a way that academia hasn't taught us, which solidified our overall engineering capabilities.

What's next for Course Concepts Detector using Google APIs

We can use the Google Streaming service to input longer videos and make a more user-friendly/beautiful user interface that can more optimally provide value to the end-user. This idea can be expanded on the industrial scale to assist employees/ users in finding important information from any video/audio they need to view.

Built With

• google-cloud
• google-text-to-speech-api
• python
• text-to-speech