Because of this pandemic, students including me and professionals across the globe are facing one problem in day-to-day life which is of online mode of work. Students are attending long hours of online classes and don't get time to take notes because of various reasons during ongoing classes, So during the time of study they find it difficult to again go through those hours of lecture videos, but if they have the facility to get the summary of this lectures through which they can understand the important context of it then this is best for them. Also for the working professionals who are attending meetings with different clients/colleagues, at the end of the day, he can't understand exactly what points are discussed during the meeting, so if he gets a summary of it then he can understand the points discussed at a greater extent.

What it does

To address this issue I made a summary generator that can take video or speech or pdf documents as input from users and generate a short summary of it. Hereby considering different aspects I have implemented these three services.

How we built it

This application builds with the help of technologies like Deep Learning, Natural Language Processing, Computer Vision, and the most important Python as the interacting language. Here I have made three services viz. Video to the summary, Speech to the summary, and pdf to summary. So these services work on a deep learning-based pretrained t-5 transformer model which is basically a sequence to sequence learning model it helps to do most of the NLP steps and generate summary from input text data. And to feed text to this model(t-5 transformer model) different libraries like google's speech_recognition to record speech and generate text,moviepy to get voice form video, pytesseract to implement OCR technique on the pdf file, and other libraries are used. Different logical tips and tricks are implemented using python to resolve the errors. This system integrates with the HTML web page through Flask framework and this project is hosted on AWS EC2 servers on DL1 instances and Habana’s SynapseAI SDK to efficiently increase the performance of the whole application and decrease the training time of deep learning models.

Challenges we ran into

As I am newly exploring the AWS instances and all these technologies, I got many errors some can easily resolve but a few take a lot of time. Because of this instance, it's really helped me a lot to run the applications because other instances can't get that much load on them.

Accomplishments that we're proud of

I feel proud because I completed the goal and mission which I have set during the start of this project. And solve a genuine problem faced by many peoples.

What we learned

There is a lot of lessons I have personally learned during working on this project. As different technologies are implemented and different scenarios are considered, various ideas are coming, different errors are taking much more time and patience. My coding skills are improved along with it I get to know about many AWS services which definitely help me in the future.

What's next for Summary Generator

We can implement this idea in hardware also, we can integrate this with AI assistants like Alexa and Siri, so let's take an example: We are setting on meeting room so we can turn on the assistant it records the whole discussion and generate a summary out of it, which helps to revise the important points which are discussed dung the meeting.

Built With

Share this project: