Kami

This web and mobile app uses OCR and NLP to extract text from images and summarize them to make taking notes and reading long articlesm easy.

Comment

Inspiration

Nowadays, with the vast amount of information available on the internet, if you want to learn, review, or analyze something, it takes far too long. Almost everything we read stretches out the important points into long, winding paragraphs full of prose and unnecessary filler. If only there was a way to filter out the unimportant text...

What it does

Kami uses Machine Learning to adaptively summarize inputted text. You can input by copy and pasting text or submitting images with text, and your previous scans will be saved on the cloud for future access. It uses NLP and OCR backends built in Flask and integrates with web and mobile apps built in AngularJS and Flutter.

How I built it

Kami has two distinct portions: a front end and back end. The backend is written in Python, using the Flask web server wrapper. Our front end is cross platform, enabling users to use Kami on virtually every device they choose. We currently offer a web application and iOS/Android applications, all following the Material standard by Google. Our apps offer an easy to use interface to cut down on the filler in your text, written in AngularJS and Flutter.

Our back end performs two important tasks, converting the image input into text and summarizing through NLP.

The first step of this process is OCR. This is done by upscaling the input images with OpenCV, filtering, thresholding, and sharpening, then using the Tesseract OCR (Optical Character Recognition) engine to convert to text.

The second step of the process is NLP. This breaks down the paragraph into its fundamental structure (clauses). Through the usage of the Term Frequency and Inverse Document Frequency algorithms, each clause is given a ‘weight’ (based on the words in it). If the weight of a clause is above the threshold, it is included in the summary. We hosted our API on a production AWS EC2 Ubuntu server. We used Gunicorn to convert our development Flask web server into a production web server with workers.

Challenges we ran into

Our EC2 server was on the free tier. While it allows us to serve HTTPS API traffic to anyone, its low specs bottleneck us, significantly throttling the performance of the OCR engine. Our performance was significantly better on a mid-range desktop on localhost. We also dealt with issues with the CORS protocol. We also had issues with OCR's handling of dark-colored pictures. Our text summarization algorithm was throwing errors when nonsensical text was inputted - this problem still exists.

What's next for Kami

We're really proud of Kami, and we like to think that we made something that people can use. However, if we could, we'd love to expand the capabilities of our OCR and make apps more versatile. We'd also love to upgrade our server to allow us to use the full capabilities of our code. We'd also like to create an Electron desktop app.

Share this project:

Updates