Math Tutor AI Agent

Math Tutor AI Agent is a multimodal tutor that guides students step by step through mathematics problems, explaining the reasoning at every stage.

Comment

Inspiration

The idea for Math Tutor AI Agent came from a conversation that many of us have had, or witnessed, or lived through. A student bright, motivated, working hard sitting at a table late at night with a maths problem they cannot crack. No teacher available. A YouTube tutorial that explains the general case but not the specific confusion. A parent who studied a different curriculum twenty years ago. And an AI tool that, when asked for help, simply produces the answer. That last part is the one that bothered us most. The student copies the answer. Gets the homework mark. Learns nothing. The next test comes and the gap is still there, or wider. The students who do well in mathematics have, almost universally, had access to someone who could sit with them and think through problems together a patient parent, an older sibling, a private tutor. That access is not evenly distributed. It correlates almost perfectly with income and geography. A student in Lagos, in Dhaka, in São Paulo has the same intellectual potential as a student in London or Singapore. What they often lack is not ability, it is access. We built Math Tutor AI Agent because we believed that a well-designed AI system could close that gap. Not by giving students answers, but by doing what a good tutor actually does: guiding, questioning, explaining, and adapting to the learner in the moment. The inspiration was not a technology problem. It was an equity problem. The technology was how we chose to address it.

What it does

Math Tutor AI Agent is a multimodal AI tutoring system that teaches students how to solve mathematical problems step by step, rather than simply producing answers for them. Students can submit problems in three ways: by typing an equation or word problem, by uploading a photograph of handwritten homework or a printed worksheet, or by speaking the problem aloud. Once submitted, the agent identifies the problem type, breaks it into logical stages, and walks the student through each one, explaining the reasoning at every step. The defining feature is interruptible tutoring. At any point during an explanation, a student can ask a follow-up question. The agent pauses, addresses it fully, then resumes exactly where it left off. The system also generates practice questions, renders all maths beautifully with KaTeX, streams responses token by token via WebSocket, and maintains full session memory across a 30-minute tutoring session.

How I built it

The backend runs on Node.js and Express, with WebSocket support for real-time streaming and REST endpoints for each input type. An AgentService layer orchestrates all calls to the Gemini API and manages session memory. The AI model is Google Gemini 2.5 Flash, chosen for its native multimodal capability across text, image, and audio in a single API call. The frontend is React and Vite, with KaTeX for client-side maths rendering. Deployment is fully automated to Google Cloud Run in us-central1, containerised with Docker, with the API key stored in Secret Manager. The service scales to zero when idle, keeping the entire running cost within the always-free tier.

Challenges I ran into

The hardest challenge was making the agent teach rather than answer. A language model naturally wants to give the most complete answer immediately. Overriding that required many prompt iterations, the breakthrough was reframing the instruction from "do not give the answer" to "your goal is for the student to reach understanding themselves." The shift produced fundamentally different tutoring behaviour. We also had to rebuild the communication layer mid-project, migrating from stateless REST to WebSocket when we realised REST couldn't support the conversational continuity interruptible tutoring requires. Handwritten image input surfaced non-standard notation edge cases we solved by having the agent state its interpretation before proceeding. Tool-calling calibration required rewriting tool descriptions to prevent the model invoking the calculator for arithmetic it could handle natively.

Accomplishments that we're proud of

The interruptible tutoring flow works exactly as designed and it feels natural. When we first tested a student interrupting mid-explanation and watched the agent pause, address the follow-up, and resume from the exact right place, that was the moment the project became real. We are also proud that the system is genuinely multimodal from day one, that the deployment is fully production-grade and publicly accessible, that the maths rendering is beautiful enough to feel like a legitimate educational tool, and that the entire system runs at zero cost within the Google Cloud free tier.

What I learned

Prompt engineering is product design. The most impactful work was not writing code — it was writing the system prompt. The way you frame an instruction to a language model determines the entire character of the experience it produces. Designing for access rather than features sharpened every decision. Real-time streaming changes the psychology of an interaction, watching an explanation build token by token feels like watching someone think, and that quality matters enormously for a tutoring application. And the combination of serverless architecture and AI APIs means a production-quality tutoring system can be deployed at essentially zero cost, the infrastructure barrier that has historically made educational technology inaccessible no longer exists in the same way.

What's next for Math Tutor AI Agent

Persistent learning profiles across sessions. Adaptive difficulty that responds to demonstrated understanding. Expanded coverage into physics, chemistry, and computer science. Interactive graphing via Desmos integration. A lightweight offline mode for low-bandwidth environments. And a student community layer that turns an individual tool into a collective resource. The goal has not changed from the first line of code to the last: every student, everywhere, deserves a patient and knowledgeable tutor. We built the first version of that tutor this week. We intend to keep building it.

Built With

Share this project:

Updates