Covid 19 affected everyone and what can be more difficult than preparing your exam through Online materials with long hours and couldn't get a summary of lecture notes? It can be difficult to digest so much content in a limited amount of time, especially during the midterm or exam period.
Introducing Summarai ! Summarai leverages machine learning and Natural language processing to allow students to capture quick summaries of lecture notes from learning videos to improve productivity and their learning experience. We wanted to develop something to support student productivity, celebrate learning diversity, and help students to destroy their exams like an unbowed Samurai!
What it does
Summarai is a tool that forges a magnificent katana from a youtube video so that you can fight like a samurai to destroy your exam. With the power of computer vision and Natural language processing, Summarai forges a katana sword(pdf file) that is filled with fire and blood(summary of the video contents with a screenshot of the image). A knowledge tree graph is also generated which provides a quick overview of the contents of the entire lecture video and a list of recommended video links will also be published to the pdf file.
How we built it
Challenges we ran into
While using Keras and CNN model in each image to perform text recognition was great, we struggled to get a better summary of each keyframe of the video lecture. We spent more time on how to use entity sentiment analysis from Google NLP API to get a better summary of audio along with Google Cloud Vision API to perform text recognition to improve the individual summary of lecture slides. result of text summary.
Accomplishments that we're proud of
We were extremely proud of our overall architecture and how our machine learning-related components got to provide a great result to improve student's productivity. We were also proud of our front-end which is user-friendly and colorful for the users to get a better learning experience while preparing for exams or studying in general.
What we learned
Our team learned how to use a fast API and various open-source APIs in general. We as a team learned more about neural networks and Keras as a framework and we got to learn a lot about cloud vision APIs and OpenCV on how to handle image processing and text recognition in general.
What's next for Summarai
We can add features such as a link for a specific keyword to give the users a better understanding of the context. We also need to improve the performance of our NN model so that we can reduce the dependencies from Google Cloud Vision API. In addition, we can also add more components to improve the overall throughput and reduce the latency of the training process with other cloud computing services.