OCR, though developed, but still not a popular part of our daily life. Also, the idea of making machines do the task that we humans do is itself fascinating. Here, in this hackathon, the challenge of reading from images and evaluating the operations in images that we humans perform is indeed worth brainstorming.
What it does
Our model reads an input image and analyzes the expression and decides whether it is an infix/prefix/postfix expression. Also, it evaluates the value of the expression shown with high accuracy and speed.
How we built it
We first prepared a dataset to train our model to classify characters as appropriate numbers or operators and trained it using CNN. Then, we identified the position of the operator on basis of which we decided whether it is an infix/prefix/postfix expression. And then according to the operator and numbers detected, we evaluated the expression to give the desired result.
Challenges we ran into
The biggest challenge that we encountered was to train our model to classify the character as 0 or 1 or 2 or..so on without using an external dataset (as restricted in question). We overcome this challenge by working on over 50K images that were given to us. we took around 200 images of each class, split those images into three parts, and manually separated them into folders(classes) of 0, 1, 2, ... and so on. Quite a tedious task, but yes, we were finally able to prepare our own dataset, and then using it we trained our model to categorize the detected character as appropriate number/operator(whether it is 0 or 1 or 2 or + or...). We also used the concept of data augmentation to further improve the accuracy of our model.
Accomplishments that we're proud of
We are able to train our model to give an accuracy of 98% in the first task and 90% in the second task when tested on the dataset provided (which was initially kept aside for testing purposes). Also, applying CNN to train the model is something we are really happy about as we went out of our zone of using standard training models (eg, logistic regression, random forest regression, etc.) and were actually able to apply this deep learning model.
What we learned
Firstly, we developed upon our skills of becoming future data scientists and learned a lot while working on this project as we actually got to apply the theory and combined many aspects of our learning. Secondly, collaborating and working in a team together helped us to actually improve our skills and learn from each other itself.
What's next for Operation HAY
Doing this project together was more of a fun activity for all of us rather than a typical "hackathon" as exploring this field and executing these machine learning and deep learning models was really exciting and thrilling and we will not be stopping here. We are definitely going to explore furthermore in this field and will pick up more projects. One of the projects which we guys have planned is detecting doctor's writing from prescription and will soon start working with it.