NoNotes RPA

Inspiration

Most people hate taking notes in class. Many times, it's difficult to keep paying attention to what is being said while writing notes at the same time.

We thought it'd be liberating if students could go to class and follow along with the professor without having to worry about writing down everything and being afraid of missing something important. That time and effort could be put into actually following what your prof is teaching all while knowing your lecture notes are being prepared for you to review in nearly real time.

So we wanted to come up with a way to transcribe videos of lectures instantly, even allowing us to SEARCH lecture recordings and other educational videos for their content.

If you want find yourself forgetting what a professor said about "AI," or "quick sort algorithms," this hack is JUST what you need!

What it does

Processes lecture video or audio and transcribes speech into text using the Google Cloud Speech-to-Text API. This lets you SEARCH THE LECTURE WITH CTRL+ F!!

Implementing:

• ffmpeg for video audio extraction and audio conversion to single channel .flac audio files
• Google Cloud Speech-to-Text API
• Google Cloud Buckets

How We built it

Using Google Cloud APIs

• Speech-to Text
• Cloud Bucket
• As well using a package called ffmpeg for video/audio handling

Challenges Faced

PLENTY ! Couldn't unpackage library properly to connect to the google cloud API, this took us ALL day to figure out. Eventually, we used the gcloud SDK directly to connect to the Cloud Speech-to-text API, got instant results which was the best feeling ever.

We also learned how to USE UiPath as originally planned, however we decided to create our own hack instead of leveraging UiPath to connect other web services that transcribe videos.

Accomplishments We're Proud Of

We are most proud that we were able to achieve our goal, transcribing videos. We are incredibly impressed by the accuracy and ease of use of the API once we figured out how to set it up and use it.

We learned how to use the Google Cloud platform.

What was Learned

How to use these incredible technologies: Google Speech-to-Text API Google Cloud Bucket UiPath - Not used but we learned :)

What's next for NoNotes RPA

Text Summarization Quickly summarize the transcription into short paragraphs of the video

NoNotes Mobile App Automatically summon the hack from your phone and instantly transcribe your videos

Google Internship :P What if all videos across the web had metadata of their transcription, the search algorithms would be much more superior and direct better content to users based upon their queries. You will be guided results based upon the CONTENT of the video, what is SAID and done, not just search by Title/Description/Tags, this will help curious individuals reach better suited/ relevant content.

Built With

• google-web-speech-api
• uipath