-
-
MARGIN
Margin — Gemma 4 AI Mathematics Notebook & Conversational Tutor
A maths notebook. A tutor in the margin.
Margin is an AI-powered mathematics notebook that lets students solve problems naturally on an infinite digital whiteboard. It recognizes handwritten equations, checks mathematical steps using SymPy, generates interactive graphs, and provides spoken explanations through a conversational AI tutor.
Our goal was simple: make solving maths on a computer feel as natural as writing in a notebook, while giving students the guidance of a personal teacher.
Inspiration
Most of us have spent hours solving maths problems in notebooks, writing equations, crossing out mistakes, and trying different approaches until something finally makes sense.
But there's one problem with traditional notebooks. They can't tell you when you've made a mistake.
Imagine solving a differentiation problem and accidentally writing:
$$ \frac{d}{dx}(x^3)=2x^2 $$
instead of:
$$ \frac{d}{dx}(x^3)=3x^2 $$
That one small mistake could affect every step that follows. And unless someone checks your work, you might not realize it until much later.
We noticed that existing digital mathematics tools often solve one problem while creating another. Some require students to type complicated equations, learn LaTeX, or constantly switch between a calculator, a notebook, and an AI chatbot.
We started wondering why technology couldn't adapt to the way students already solve mathematics.
What if you could simply write an equation and your notebook understood it?
What if it could notice a mistake, explain what went wrong, draw a graph, or even write the next step alongside you?
And what if you could ask it questions just like you would ask a teacher sitting next to you?
That's where the idea for Margin came from.
We wanted to create something that combined the freedom of a physical notebook with the intelligence of an AI tutor, without interrupting the natural process of learning.
What It Does
Margin brings handwriting, mathematical verification, interactive visualization, and conversational teaching into a single workspace.
Natural Handwriting Recognition
Students can write mathematical equations directly on an infinite whiteboard using a mouse, stylus, or touchscreen.
There's no need to type equations manually or learn complicated mathematical formatting.
Margin uses multimodal vision models to interpret handwritten expressions and convert them into structured LaTeX.
We also provide different canvas styles, including grid, lined, and dotted backgrounds, so students can choose the workspace they're most comfortable with.
Step-by-Step Mathematical Verification
One thing we wanted to avoid was an AI that confidently explains an incorrect answer.
While language models are excellent at understanding questions and explaining concepts, they can sometimes make mistakes during mathematical calculations.
To address this, we integrated SymPy, a Python-based symbolic mathematics library.
Margin uses SymPy to verify supported mathematical operations, including algebraic simplification, differentiation, integration, factoring, and equation solving.
For example, when a student writes:
$$ f(x)=6x^2 $$
Margin can verify that:
$$ f'(x)=12x $$
Instead of relying entirely on the language model, we use a combination of AI reasoning and symbolic computation.
The AI handles recognition and explanation, while SymPy provides an independent way to verify supported calculations.
Conversational AI Tutor — Teacher Ava
We didn't want Margin to feel like another chatbot that simply returns answers.
We wanted students to feel like they had someone teaching them while they worked.
That's why we created Teacher Ava, a conversational AI tutor who explains mathematical concepts using natural speech.
Students can ask Ava to explain a step, provide a hint, simplify an explanation, or help them understand why a particular method works.
For example, instead of reading mathematical notation mechanically, Ava can say:
"The derivative of six x squared is twelve x. We multiply the coefficient by the power and then reduce the power by one."
We also developed a mathematical speech-conversion system so that equations are spoken in a way that sounds natural.
The idea is to make learning feel like a conversation with a patient teacher who is always available when needed.
AI Whiteboard Co-Writer
This is one of the features we're most excited about.
Margin doesn't just generate answers in a separate chat window. It can write directly on the whiteboard alongside the student.
When someone gets stuck or completes a problem, the AI can use a virtual pen to write the next mathematical step or final expression.
The writing appears smoothly on the canvas, creating the feeling that someone is solving the problem with you.
It also keeps the entire solution in one place, so students don't need to move between their work and a separate explanation window.
Interactive Graphs and Visual Explanations
Sometimes an equation makes much more sense when you can actually see what it represents.
Margin can display Cartesian coordinate graphs alongside mathematical expressions.
For example, students working with:
$$ y=x^2 $$
can see how the equation produces a parabola and explore the relationship between the mathematical expression and its graph.
We designed the workspace around three organized sections: equations, explanations, and graphs.
This helps students follow the solution without different elements overlapping or cluttering the whiteboard.
Need Any Help? — Socratic Tutor
We also added a small, dockable assistant panel in the bottom-right corner of the workspace.
Students can open it whenever they need help and ask questions through text or voice.
Rather than always giving away the final answer, the tutor can guide students through the problem using hints and follow-up questions.
We wanted Margin to encourage students to think for themselves while still giving them support when they need it.
How We Built It
Margin is a full-stack web application built around a real-time digital whiteboard and a multimodal AI pipeline.
Frontend and Whiteboard
We used Next.js 14, React 18, Tailwind CSS, HTML5 Canvas, and Lucide Icons to build the interface.
Since handwriting is central to the application, we spent considerable time improving the drawing experience.
We implemented quadratic-curve stroke smoothing to make handwriting feel more natural, even when using a mouse.
KaTeX handles mathematical notation and renders equations clearly throughout the workspace.
AI Vision and Handwriting Recognition
For handwriting recognition, we integrated Google Gemma 4 and a Gemini Vision cascade using gemini-3.1-flash-lite and gemini-3.5-flash.
The vision pipeline analyzes handwriting and converts recognized mathematical expressions into structured LaTeX.
We also developed specialized prompts to help distinguish mathematical symbols that look similar when handwritten.
Mathematical Verification
We built a local Python microservice using FastAPI and SymPy 1.13.1.
This service handles symbolic operations such as differentiation, integration, factoring, equation solving, and equivalence checking.
Separating mathematical verification from the language model helps make calculations more reliable.
Voice and Conversational Teaching
For Teacher Ava, we integrated Microsoft Edge-TTS using the en-US-AvaNeural voice.
We developed a mathematical text-to-speech preprocessing system that converts mathematical expressions into natural spoken English before generating audio.
This allows Ava to explain equations in a conversational way instead of reading raw LaTeX or mathematical code.
Database and Persistence
We used SQLite with Prisma ORM to save notebooks and support automatic persistence.
This allows students to return to their previous work without having to manually save their progress after every change.
Challenges We Faced
Building Margin involved several challenges that we hadn't fully anticipated when we started.
Making AI Understand Handwritten Mathematics
One of the hardest parts was getting the vision model to recognize mathematical handwriting accurately.
A handwritten (x) can look like a multiplication symbol. A plus sign can resemble the letter (t). Exponents and subscripts can also become difficult to distinguish.
For example:
$$ x^2 \neq x_2 $$
These expressions look similar when written quickly, but they mean completely different things.
Initially, we also noticed that converting canvas strokes into images sometimes caused small gaps and loops inside characters to disappear.
This made certain handwritten symbols harder for the vision model to recognize.
To improve the results, we developed a custom rasterization pipeline that identifies the region containing handwriting, applies padding, and preserves stroke clarity during image conversion.
We also adjusted our character-disambiguation prompts to make better use of mathematical context.
These improvements made handwriting recognition more consistent.
Reducing Recognition Latency
Our first implementation captured large screenshots of the whiteboard and sent them directly to the AI vision model.
The problem was that these images could be around 4 MB in size.
Repeatedly transmitting such large payloads introduced noticeable delays, especially when students were writing continuously.
We realized that most of the screenshot was unnecessary because the actual handwriting occupied only a small part of the canvas.
So we developed a bounding-box rasterizer that crops the written region, normalizes the image dimensions to a target edge of 720 pixels, and exports the result as an optimized JPEG.
This reduced typical payload sizes from approximately 4 MB to less than 40 KB.
The smaller images significantly reduced transmission overhead and helped make recognition more responsive.
Preventing Overlapping Voice Responses
Another unexpected challenge came from Teacher Ava.
When students wrote quickly, multiple automatic scans could trigger several text-to-speech requests almost simultaneously.
Sometimes the first audio response would start playing just as another request finished.
The result was multiple voice explanations playing over each other.
We solved this by building a centralized speech manager using monotonic generation IDs and AbortController.
Whenever a new speech request begins or the student starts writing again, the previous request is canceled and existing audio playback is stopped.
This ensures that Ava focuses on the latest interaction instead of speaking over herself.
Stopping the AI From Analyzing Its Own Writing
This was probably one of the most interesting bugs we encountered.
After implementing the AI whiteboard co-writer, we noticed that the handwriting recognition system was detecting the equations written by the AI itself.
The application would analyze those equations, generate another response, write something new, and then analyze that new writing again.
Essentially, we had created an unintended feedback loop.
To fix this, we introduced unique identifiers for AI-generated strokes using the ai_stroke_ prefix.
The recognition pipeline now distinguishes between student handwriting and AI-generated writing.
Only student strokes are included in automatic handwriting analysis.
It was a relatively small change, but it made a huge difference to the stability of the application.
Accomplishments That We're Proud Of
Making Digital Mathematics Feel Natural
Our biggest accomplishment is creating a workspace where students can simply start writing.
They don't have to learn LaTeX, insert equation blocks, or interrupt their thinking to use the application.
We wanted the experience to feel as close as possible to solving mathematics in a physical notebook.
Building a Tutor That Explains Mathematics Aloud
Teacher Ava adds something that we felt was missing from many digital mathematics tools.
Students can listen to explanations while looking at their equations and following the steps on the whiteboard.
This makes the experience more interactive and gives students another way to understand difficult concepts.
Combining Generative AI With Deterministic Mathematics
We're particularly proud of integrating symbolic mathematical verification instead of relying exclusively on AI-generated answers.
Using Gemma and Gemini for understanding and explanation, together with SymPy for mathematical verification, gave us a more dependable foundation for teaching.
Bringing Multiple Learning Tools Together
Margin combines handwriting recognition, mathematical verification, interactive graphs, spoken explanations, AI handwriting, and notebook autosaving in a single application.
Getting all these components to work smoothly together was one of the biggest engineering challenges, and seeing the complete experience come together has been extremely rewarding.
What We Learned
One of our biggest takeaways was that building an AI application involves much more than selecting a powerful model.
The quality of the complete experience depends on how well every component works together.
We learned how much image preprocessing affects handwriting recognition. Small differences in stroke thickness, cropping, padding, and image resolution can significantly influence how accurately a model reads handwritten equations.
We also learned how challenging asynchronous systems can become when multiple processes are running at the same time.
Handling continuous handwriting, automatic recognition, AI-generated strokes, and voice responses required careful synchronization.
Another important lesson was understanding the limitations of generative AI.
Language models are useful for interpreting handwriting, adapting explanations, and answering questions, but mathematical verification benefits from specialized symbolic tools.
Combining these approaches helped us build a system that supports both mathematical reasoning and conversational teaching.
Most importantly, this project changed how we think about educational technology.
We realized that students don't always need another tool that gives them answers faster.
Sometimes they need a tool that helps them slow down, understand what they're doing, identify mistakes, and build confidence in their own thinking.
That's what we wanted Margin to do.
What's Next for Margin
We see Margin as the foundation for a much more interactive way of learning mathematics.
Geometry and Shape Recognition
We want students to be able to draw triangles, circles, coordinate axes, and vectors directly on the whiteboard.
Margin could recognize these shapes and help calculate angles, distances, areas, and other geometric properties.
For example, a student could sketch a triangle and ask why two of its angles are equal.
Highlighting Mistakes Directly on the Whiteboard
One of our next priorities is step-level mistake detection.
Instead of simply telling students that an answer is incorrect, we want Margin to identify the exact mathematical step where the mistake occurred.
The AI could highlight that section using red ink and guide students toward correcting it themselves.
For example, rather than immediately revealing the answer, Ava might ask:
"Look at the second step again. Did you apply the exponent rule correctly?"
We believe this kind of feedback could make learning more meaningful.
Collaborative Classrooms
We also plan to explore shared whiteboards where teachers and students can work together in real time.
Teachers could demonstrate problems, students could attempt their own solutions, and Margin could provide additional explanations when someone gets stuck.
Local and Offline AI
Another direction we want to explore is running lightweight Gemma vision models directly on students' devices through WebGPU.
This could reduce dependence on cloud services, improve privacy, and make the application more accessible in environments with limited internet connectivity.
Final Thoughts
When we started building Margin, our goal wasn't to create another AI tool that simply solves mathematics problems.
We wanted to create something that actually helps students learn.
A place where students can write naturally, make mistakes without feeling discouraged, ask questions whenever they're confused, and understand why a mathematical solution works.
Because getting the right answer is important, but understanding how you reached that answer matters even more.
If Margin can help a student understand a concept they've been struggling with, even if it's just one difficult equation, then we're moving in the direction we hoped for.
That's what Margin is all about.
Margin A maths notebook. A tutor in the margin.
Log in or sign up for Devpost to join the conversation.