Education is fundamental to the progression of society and currently, the demand for online and remote teaching resources is high. Many people may not be optimally equipped with products such as tablets and styluses so working from paper is the only viable solution. The quality of internet connection and readability of handwriting can reduce the ability to learn and teach. Papersnap provides a quick, simple, and easy solution to this problem, allowing for a better educational experience for all.
What it does
This full-stack app takes a photo of your handwritten notes and even mathematical equations before turning them into mark-up language format to render them onto a communal whiteboard on the web. Each user submission would be rendered as a resizable, draggable box on a web canvas and anyone can contribute.
How we built it
Flutter - Mobile app to take photos and send it to our server (which is connected to Azure Cognitive Services).
React and Express.js - Full-stack React web app to handle images, optical character recognition, and database with MongoDB
Python - Optical Character Recognition (OCR) for text recognition using machine learning from images and processing of the .json files outputted
HTML - Embedded XML within React
CSS - Styling within React
Challenges we ran into
The OCR algorithms supported less text than expected. Removing data from the database when deleting a text box was difficult to implement without encountering timing issues. Combining mobile and web development is new to all of us.
Accomplishments that we're proud of
Producing a finished and genuinely useful project in just 36 hours is an achievement in itself. Two of our team had never partaken in a hackathon and working remotely proved to be a challenge when helping each other.
What we learned
How to use APIs, set up a RESTful API, use databases, and implement a full-stack solution.
What's next for PaperSnap
Improving image processing algorithms to increase confidence intervals in our OCR algorithm, as well as the support of vectorization of hand-drawn diagrams to render on canvas. Increasing the number of characters that are detectable would also prove useful to some people, and a feature to solve basic maths equations autonomously would be interesting to implement. Improving workflow in BackEnd to allow seamless data flow across all platforms. Moving towards training our own algorithm.