HawkMath

Inspiration An easier, lazier way to write LaTeX. Those longer equations in calculus class are annoying to type, but what if you didn’t have to? You’ve got our HawkMath to do it for you. The true purpose isn’t only to help lazy students, but also those who are hearing impaired, lack the fine motor control to precisely type expressions, or even more. Our project helps them keep up with the rest of the students.

What it does We have the user speak into our HawkMath. It converts the speech into LaTeX, which breaks down and shows you each line that creates the math equation.

How we built it We built HawkMath using Flask for our server code, wrote our custom parser in Python, and for our front end, we used vanilla HTML and JavaScript. We used Whisper for speech -> text transcription, and then OpenAI's GPT-4o as well as our own custom rule based parser for text -> latex parsing.

Challenges we ran into Creating a custom parser was harder than expected. We ran into overlapping issues regarding division in the code. There were certain conditions which were hard to implement on multiple words.

Accomplishments that we're proud of We’re very proud of being able to push out a completed project in just under 24 hours! Although our basic parser needs much work, we are proud to have a functional, polished project.

What we learned We learned about some natural language processing techniques and the difficulties and challenges of writing a parser. Next time, we know what approach to take to correctly and properly implement a parser from scratch.

What's next for HawkMath We intend to first improve our own custom parser, as opposed to using the GPT one. In the future, we may consider implementing next-step prediction so that HawkMath could also display the results of expressions, equations, or problems shown.

Built With

• css
• flask
• gpt-4o
• html5
• javascript
• python
• whisper