Inspiration
Math tools usually focus on the final answer, but one of the most important things is learning through the steps that you take and getting to the right answer. However, imagine you are doing a problem, and you get to the final answer and check whether it is right or not, and it turns out it was wrong. You do not know where you went wrong or what steps made you lead to the incorrect answer. That is the problem that Mathbuddy solves. As you write each step of the problem it checks whether that it the right step towards to the final answer. If that step is wrong, it leads with a small hint that students can use. If you are stuck on a step there is a hint button that walks you through on how to continue. This is why I made this workspace where students can solve problems, learn where they went wrong, and learn from those mistakes.
What it does
Mathbuddy lets students write out math problems one step at a time using a mouse, stylus, or touchscreen. It automatically transcribes their handwriting after a short pause, then lets them check whether each step is mathematically valid. When a step is correct, a new writing box appears automatically. The previous equation stays visible inside the next box so students can keep working without scrolling back. Mathbuddy also recognizes when a student has reached the final answer and offers progressive hints only when requested. If you have any questions, you can simply pull up the Gemini chat on the side and ask any questions you have about the math problem.
How we built it
I built the frontend with React and TypeScript, using SVG drawing canvases for handwriting input and a Python FastAPI backend to coordinate recognition and feedback. I split the AI workflow into two parts. GLM-OCR runs locally on the GPU at 4-bit precision to transcribe handwritten mathematics. Google’s Gemini API handles mathematical step checking and hints, using the original problem and previous steps as context. Both simple and complex problems go through Gemini.
Challenges we ran into
Handwritten math is difficult to interpret because small differences in symbols, exponents, and bounds can change the meaning of an entire expression. Running OCR locally also created memory constraints. I moved to a 4-bit model and limited resource usage to make it practical on a laptop GPU. Another challenge was keeping automatic transcription responsive without sending a request after every stroke. We added a pause before recognition, reused results for unchanged handwriting, and prevented outdated responses from overwriting newer work.
Accomplishments that we're proud of
I built a working flow from handwritten input to transcription, mathematical feedback, and optional hints. I am especially proud of making the interface feel like a focused writing space. Shared drawing controls, automatic step creation, compact completed steps, and an embedded reference to the previous equation help students stay focused on solving the problem. I also kept handwriting recognition local while using Gemini to provide contextual mathematical feedback.
What we learned
I learned that recognizing mathematical notation and understanding mathematical reasoning are two separate problems. Giving each task its own part of the system made the workflow easier to improve. I also learned that AI feedback needs careful validation. A convincing explanation can still be wrong, so testing mathematical edge cases matters as much as testing the software. Finally, I learned how much small interface decisions affect the learning experience. Showing less text and giving help only when requested makes more room for independent thinking.
What's next for Mathbuddy
I want to improve the handwriting aspect of math buddy across a variety of writing styles. I want to make it so that as the student solves more questions and writes in the workspace it learns the type of handwriting the student has. I also like to plan out being able to use Mathbuddy for more complex questions like some double and triple integrals as well as line integrals.
Log in or sign up for Devpost to join the conversation.