We began thinking about this idea the night before the competition, while we were all frantically trying to finish homework since we would be at Calhacks over the entire weekend. Additionally, a few members of our team have ADHD, and focusing for extended periods of time can be difficult. We looked at some other websites that claimed to have similar functionality, but they often failed to effectively summarize anything beyond a basic news article.
Our software takes in an image, and after parsing the layout of the document, it creates a summary of the document using custom models that we made with Cohere's API, so that a more accurate summary can be created depending on the subject and type of content.
We used a library called LayoutParser to identify relevant text from the submitted image, and we took advantage of some publicly available datasets and Cohere's Finetune feature to create our own models for different types of documents. The website itself was programmed using HTML, CSS, and Vanilla JS integrated with Flask.
This was our first time creating anything with this sort of functionality, so integrating the front-end and back-end was new to us. Understanding and reformatting the datasets was also confusing at first since we don't have much experience with data processing, but we were eventually able to get existing datasets into the format that we wanted.
We're really happy about everything, since it's our first hackathon and project of this kind in general. There's a lot we could do better, but we're proud that we were able to actually have something to submit.
We learned a lot about frameworks like Flask to integrate the back-end with the front-end of the website, and we learned a lot about data processing. Cohere's API also introduced us to the field of NLP, and some of the advanced functionality that powerful models can offer.
We would like to create more models for other subjects and document types that we and our friends regularly deal with, like documents that are in old english or wikipedia/wikimedia pages.
Log in or sign up for Devpost to join the conversation.