The problem we chose to solve with this creation was something we, as CEGEP students, have encountered far too many times. Searching for solutions for school problems online has been possible for many years now, but due to the expanding array of solution solving websites, it is difficult to find the right answers sometimes, especially due to the fact that some of these sites are blocked by paywalls. The solution to this problem is our website.
What it does
Our website, SnapSolve, whose name is a play on the word “snapshot”, allows users to input problems from various different subjects as image files. Our program will then take the text from the images and search for the answers on google. The website will then return the first answer on google not blocked by a paywall. On top of this, we have an interactive chatbot (using ML) that could be adapted in the future to help students learn more about the questions that they have.
How we built it
To build our website, we first had to use tesseract’s OCR in order to get words out of images. After that, we had to clean our data through Python’s Levenshtein module. With the newly cleaned data, we use Google’s Search API to return the top 10 results of the search. After that, with the links, we used a web crawler and essentially HTML pathing to find the body paragraphs (best answers) we were looking for. After having done both ends, we finally had to implement/connect Python into HTML which required us to use flask. Finally, we used tensorflow and keras and machine learning to implement a chatbot into our website, effectively training it with hundreds of examples.
Challenges we ran into
As a collective team, we encountered a handful of issues with connecting our front end with our back end. Before the hackathon, our group did not have much experience if any when connecting different programming languages for a product, not to mention for a website. Specifically, python and HTML do not mesh well, and we were not able to efficiently connect. After not being able to use Django, which was the recommended framework of multiple mentors, we turned to using flask, which has enabled us to semi-efficiently run our website for the time being.
Accomplishments that we're proud of
We’re incredibly proud of being able to incorporate multiple different APIs into our code. None of us in our group have ever had experience working with APIs, yet all of us were able to gain a ton of knowledge throughout the 24 hour experience. Using both the internet and the mentors, we were able to efficiently use googleSearch and Tesseract, as well as a newfound knowledge and appreciation for keras and tensorflow. Even if our product wasn’t as refined as we imagined it at the start, we are all incredibly proud of the headway that we made in learning more about being a software developer.
What we learned
As mentioned previously, over the hackathon we gained a lot of knowledge in how to apply APIs and build code with a real life proper application. On top of this, all four members of our group faced a lot of challenges, and through these challenges we all learned a lot about how to search for help on the internet. Through using the internet as well as the mentors, we found that our programming ability increased tenfold per person.
What's next for SnapSolve
In the future, we would love to develop and clean up our website in order to make it more appealing to visitors. Currently, we are mostly backend programmers (using python exclusively), and do not have much collective experience on frontend to make our website visually appealing within the timeframe of 24 hours. Plus, we see tonnes of potential in bringing our website into an app format as phones have incredible potential for apps that require OCR through images (uploaded or taken). We would also love to add more data to our chatbot, as it would be able to function more efficiently and act like a real person across the computer screen.