Throughout high school and college, students have struggled with submitting math problems into search engines because they might not know what symbol to use for an exponent or a radical. Some apps gave you the option of just pressing a button that represents the symbol so you wouldn't have to type it. What if you could just write like normal and if you run into trouble with a question you didn't have to struggle with typing it? Now you can because we have taken away the need to type an equation with funky mathematical symbols.
What it does
The Synaptics touchpad takes in input from the user by monitoring the position of the writing utensil. It then uses the gathered locational data to create a physical image which is then processed using the Google Cloud Vision API in order to identify the text. Once the equation is made, it is fed through Wolfram Alpha's Computational API and which is then answered and displayed to the user.
How we built it
The application was built with Python and is made up of major three parts; the user input, the data processing, and the output. The data that comes into the computer was sorted and we were able to make a list of coordinates which was used to draw the shape as it was coming in. The shape is then saved as an image and is ran through Google's Machine Learning API in order to create a text version of the shape. That text is then inputted into Wolfram Alpha's Full Results API which solved the equation. We took the outputs from both API's and the Image that was saved in order to put them in a new window which displays the handwriting, the Google API output, and the Wolfram Alpha output.
Challenges we ran into
Some challenges that we ran into comprised of saving the image after we were able to get it drawn and outputted at the same time. We also had a hard time in the beginning when we were introduced to unfamiliar hardware and software. We also didn't have much experience with the API's that we used and also not much experience with Python but we spent this week learning the basics and figuring out a real world problem.
Accomplishments that we're proud of
Most of us were first-time hackers at a hackathon and we are proud that we were able to experience so much in such little time. We had a second-time hacker that helped us figure out what to do or else we probably would not have been able to do as much as we had done this weekend. It was a great learning experience and that is what matters the most!
What we learned
We all learn every day and some of the things that we have learned over this amazing weekend include creating graphics using Pygames and cloud deployment. We learned different approaches to one problem while gaining experience in working with other people.
What's next for Handwritten Mathematical Computation
Some things that could be done next is to have the ML API recognize text more accurately and also we could also try to include a variety of symbols that would be recognized as mathematical symbols.