front page
uplode page
tracing back summary to a pdf
tracing back summary to a video

Cramberry

Inspiration

As online lectures have become more prevalent during the COVID-19 pandemic, they have taken on an increasingly important role in teaching. However, there are times when it's simply not possible to attend class in person. When this happens, students often need to spend a significant amount of time catching up by watching recorded lectures or reviewing lecture notes. This can be time-consuming and overwhelming, especially when there are a large number of videos or readings to review. That's why we're developing a new product to help streamline the process, saving students valuable time and energy . With Cramberry, you can quickly get up to speed on missed material and focus on what's most important, without sacrificing your time or sanity.

What it does

Cramberry is an interactive website that allows you to upload multiple source materials - lecture videos (in MP4, MOV, etc. format), slides, and lecture notes (in PDF format), and then summarizes all those resources at once for you. What's more, you can easily navigate the summarized text by clicking on any concept to quickly find the relevant page or time stamp where it was mentioned. For example, if the summary says 'this figure is ~', you can simply click on the sentence, trace back to the source document and get detailed information there. With Cramberry, you'll have a streamlined and efficient way to catch up on missed material, without having to spend hours watching long lecture videos or reading lengthy chapters.

How we built it

Our product is built using React for the front-end and Node.js for the back-end, which allows for efficient and seamless communication between the different components of our system. Additionally, we use natural language processing algorithms written in Python\flask to analyze and summarize the raw text data that is processed by our back-end. To ensure data security and reliability, we store users' information and summaries in MongoDB. With the cloud version of MongoDb, we have one database and severel columns to keep track of users' upload and summary history while providing them a seamless interface to intereact with it. The summarization is done based on the graph-based algorithm using google's pagerank algorithm. The graph is constrcted based on the sentences and sentence similarities of the document or text extracted from the audio/video. The pagerank algorithm is used to pick important nodes(sentences) as a summary. The final summary is refined using the ChatGPT openai API.

Challenges we ran into

Since the NLP part of our product is primarily developed in Python, while the back-end and front-end are designed in JavaScript, we initially faced difficulties transferring information between the two frameworks. However, after spending some time investigating the issue, we were able to find effective solutions and successfully integrate the different components of our product. Additionally, we had to figure out how to use cloud storage to keep our data secure and accessible, which required some additional time and effort.

Accomplishments that we're proud of

The summarization algorithm is implemented step by step instead of using the pre-trained model. The summarization algorithm is working pretty well and our front-end webiste is designed to be user-friendly and intuitive.

What we learned

Throughout our development process, we have gained valuable experience in building our own API calls and using MongoDB for data storage and retrieval. We also learned the basics of Node.js and its syntax, which allowed us to create a more efficient and streamlined development process. Moreover, our team members brought a wealth of knowledge and expertise to the table, sharing their insights and experiences with us, which helped us improve our product and develop new ideas. With these skills and knowledge, we have created a powerful and effective product that can meet the needs of our users and help them achieve their goals.

What's next for Cramberry

As part of our commitment to providing the best possible service to our users, we are constantly working to improve and enhance our product. We recognize the importance of reliability and are committed to ensuring that our service is always up and running smoothly. Additionally, we are planning to add support for more file types, which will allow our users to upload and summarize a wider range of resources. Furthermore, we are dedicated to collecting and analyzing data to improve our NLP model and algorithms, which will help us provide more accurate and effective summaries. With these efforts, we are confident that we can continue to improve our product and meet the evolving needs of our users.