AutoNote

Start Screeen

Inspiration

What if video lectures could be converted to visual printable content? We have watched and prepared from so many online lectures. But how do we use as a normal readable guide?

What it does

The application is revolutionary in its idea. I managed to convert a full lecture video to a book pattern PDF with figures and text.

How we built it

We build it using Watson's Cognitive APIs. The process is bifurcated into 2 part. A speech to text converter which uses Watson's SpeechToText API to convert the speech to transcripts with timestamps. The next part deals with video. The video is sampled at a 1s interval and send one by one to the Waston Visual Recognition API which returns a relevance of the snapshot. If the image is relevant to education, it is recognized using the classification tags returned by Watson. Irrelevant snapshots such as a picture of only the professor teaching are eliminated. The relevant pictures are sent to a clustering mechanism achieved using KNearestNeighbours algorithm to eliminated redundant images. These, in turn, are mapped with the corresponding audio timestamps and converted to PDF in python.

Challenges we ran into

The Visual Recognition in Waston can be customized to a classifier specific to Education related videos. This requires a huge training set and additional time in the measure of days to properly classify all examples.

Accomplishments that we're proud of

The app itself is our proudest accomplishment. We were able to successfully use the Watson speech to text API and the visual recognition API for speech and snapshot analysis.

What we learned

We learn the use of API uses and in general the problems and benefits of Watson in general. We got an idea of how intricately Watson can analyze speech and images.

What's next for AutoNote

We hope the system develops to an even successful venture. We would like to see it spread across different campuses to improve the life of students. It also has huge potential in the development of future Massive Open Online Courses.