Inspiration
We would be remiss if we were to say that our inspiration was single-faceted in nature. As a team of five, we all have different backgrounds and reasons for being in coding ranging from enjoying the rush of solving hard problems to the feeling when you have an impact in the world. Although we all have our own stories, the one thing that is certain is we find inspiration in doing all that we can to try to help others. Within this hackathon, we were inspired not only by the nonprofits that we are working with closely, but by the overall presence of just how many people were able to come together to do some good in the world for a weekend. We may be driven by inner cravings of problem-solving, but even more impactful is the opportunity to be a part of something that is truly meaningful in the world around us. It is this combination that truly inspires us.
What it does
Our system is a web-based application that takes in an empty paper form image and any filled images of that form and digitizes them so that all of your collected data is stored in a SQLite database, easily queryable for any of your data analysis needs.
How we built it
The Front-End of our application is running on React.js with an embedded Java Applet for the Upload New Form and Upload Completed Form functionality. The Back-End is running off of a few different things. To start, we have a .Net Core API with an SQLite Database. To handle image processing on the back-end, we are incorporating python scripts for OMR and potentially OCR through API calls to these functions.
Challenges we ran into
Throughout the development process, we encountered many different types of challenges. We originally started this project with the goal of using the Google Vision API in order to extract the text from our Paper Forms. However, after working with the data that we had access to, we realized that it wasn't completely feasible to train a model for this project especially in the time allotted.
From this, we moved on to looking into other OCR techniques as well as some ML techniques for modeling different types of questions and their answers. Again, this proved to be difficult as we didn't have enough data in order to train a reliable model.
From this step, we went back to the drawing board looking for another way that we could potentially solve the problem of deciphering a handwritten response to a survey or a short-answer form. We went through a couple more iterations including but not limited to generalizing the types of questions that could be asked, including an icon to help ensure that there was an accuracy with scanned documents in terms of rotation or translation, and even considering running OpenCV in real time on Android devices as a mobile solution.
We finally settled on a solution that took us away from the problems we were encountering with all of the ML techniques. Although there were many aspects of our current solution which were new to our team including a React.js Front-End, embedding a Java Applet into this system all designed using Java's GUI features, incorporating Python scripts in the .Net Core API that runs the system and storing data collected so that it could be communicated to other parts of the system, we've managed to come a long way from where we started and we've all learned something new throughout the process.
Accomplishments that we're proud of
Even with some of the struggles and setbacks that we've experienced over the last 36 hours, we are incredibly proud of what we've been able to accomplish as a team. We all came into this hackathon with no idea of what to expect and no team in sight. For four out of five of us, this was our first hackathon and we could never have imagined going into this process that we would come out with a viable product that could really make a difference in our own community to truly help those who need and deserve it. For those working on Front-End Development, JavaScript and specifically React.js was completely uncharted territory and we have been able to come a long way from asking for help with downloading the setup files to having a basic home structure and upload process for our application. Along with this, we have all learned a great deal about many different image processing techniques from OCR to OMR and everything in-between. In terms of Java Development, we have been able to take an idea without any concrete understanding of how to code it and transform it into a Java Applet that can potentially be ran as a service or embedded element in our application to help increase the efficiency of our solution for the NPOs. And finally being able to take all of these separate moving pieces and fit them together into a potential solution has been a true lesson in Full-Stack development that goes beyond anything we thought possible.
Although these technical accomplishments are quite awe-inspiring to us, what we are truly proud of is how we worked together. Being strangers, it is easy to imagine a situation where there was a lot of push-back or redirecting. But this was never the case with our team. We all met because we were alone looking to solve a greater cause than ourselves and we were able to come together and work towards a common goal of helping our community through innovation and determination. We had an amazing sense of communication and cooperation that is rare among teams at events like this. We were always there for each other to help solve any problems that arrised and to help cheer on through both the bad and the good. We were able to use our unique characteristics and expertise to ensure that everyone could work on something they were interested in and learn while working towards a larger end goal. The teamwork is something that will stay with us long past this hackathon's ending.
What we learned
In an effort not to be overly prolix, it is clear from the previous sections that we have learned so much through this project. To summarize some of the technical skills one might see on a resume, we have gained lessons in web development, front-end development with React.js, back-end development with .Net and incorporating scripts as services, image processing with OpenCV and it's capabilities, and overall the many steps in agile software development. Soft-skills that we have learned include but certainly are not limited to the importance of teamwork, determination, dedication, responsibility, time-management, collaboration, inclusion, and certainly patience.
If we were to sum up everything we've learned, it may be best phrased as "it takes a village." The work and progress that we have made over the last 36 hours would be impossible without our fellow teammates and certainly without the support of the mentors and volunteers that helped us keep going and think outside of the box especially when things didn't seem to be moving forward. If anything is clear, it seems obvious that given the collaborative effort of everyone, we are capable of being the change we want to see and this is certainly a lesson that resonates through the work that every team has done through this hackathon. This is the true potential of innovation for the sake of making a difference.
What's next for A^2D^2 -- All About Dat Data
- Currently, our system only handles one-page forms as only one image can be uploaded at a time. We'd like to adjust this so that multiple page forms could be accepted without manual input.
- In order to get accurate results for text extraction, we need more time to train and evaluate different OCR methods. Ideally, we would gather samples of questions and answers and look for a method that gives us the highest amount of accuracy without risking efficiency.
- To increase the security of the app, we plan to scan all files uploaded prior to acceptance for any signs of bad actors trying to gain access to the system. Also with future versions, we will look into implementing a possible sign in the system using Google Sign in as an authentication service.
- For the best convenience possible, we plan to have this application deployable through Heroku, which is a one-click hosting platform allowing for minimal effort to run your own instance of our app for your NPO.
- Lastly, we plan to have a CSV export function available through the Reporting functionality so that you can download any of the data stored from your forms for external use.
Log in or sign up for Devpost to join the conversation.