Track: Reply AI, the "AI on Campus" challenge (AI for learning).
Inspiration
Some ideas only click when you see them move. Watch a secant line swing until it settles into a tangent, and the formula for a derivative becomes a record of something you saw. Animated explanations do this well, but a video explains its author's example, not the problem on your sheet, and you cannot stop it to ask why a term disappeared.
We wanted that kind of explanation for the exact problem a student is stuck on, in a form they can interrupt, question and change. That is how we read the "AI on Campus" learning track: AI that helps a student understand, with the student in charge.
What it does
Type any math or physics problem, word problems included. VisuMath solves it, draws it as a live 2D or 3D scene, and walks you through the solution step by step. On each step the camera moves to what matters, a pointer lands on the exact spot being explained, and a short caption says what you are looking at.
It is a graph and not a video, so you can take over at any point:
- Drag a slider and every curve, number and caption follows. Shrink \(h\) and watch the slope of the secant, \(\frac{(a+h)^2 - a^2}{h} = 2a + h\), settle on \(2a\).
- Click and ask. Pick a point, a curve or a line of the worked solution and ask about exactly that. The tutor answers in a few sentences and points back at the graph.
- Talk to it. A voice reads each step aloud, and you can ask your question out loud.
- Make it yours. Add your own equations, turn and zoom the picture, and share the whole scene as a link. There are no accounts.
It also simulates what has no formula. A pendulum obeys
$$ \ddot{\theta} = -\frac{g}{L}\sin\theta $$
and the app integrates that directly. Released from 60°, its period comes out 7.3% longer than the textbook's \(2\pi\sqrt{L/g}\). Fluids are simulated live on the graphics card.
How we built it
One decision carries the rest: the model never writes code. Claude solves the problem and then writes a scene, which is plain JSON: a list of objects (28 kinds, from functions and vectors to surfaces, simulations and fluids), sliders, and the steps of the tour. Any coordinate can be an expression such as f(a + h), so the picture stays live.
Because a scene is data and not code:
- It is safe to render. The browser validates it, drops what cannot be drawn, and asks the model for one repair if too much was lost.
- It can be drawn while it is still being written, so the tour starts as soon as step 1 has arrived.
- The student can edit it, the tutor can change it, and it fits in a URL.
The app is Next.js and React. Expressions are evaluated with mathjs and drawn with three.js, in one renderer for both 2D and 3D. Motion is integrated with fourth-order Runge-Kutta, and sudden events such as a bounce are located inside a time step. The fluid is Stam's stable fluids, written as shader passes. The voice is ElevenLabs, and spoken questions use the browser's speech recognition. We built it with the help of Claude Code, and more than 200 unit tests check the mathematics as well as the code.
Challenges we ran into
- The wait. The model has to solve a problem before it can draw it. We changed the format so each step carries the objects it introduces, and wrote a parser for unfinished JSON. The student reads step 1 while the rest is still arriving.
- Trusting the output. A model can write an expression that does not evaluate, or point at an object that does not exist. Everything passes through a validator that keeps what can be drawn and reports what it dropped.
- Labels in 3D. Labels are text laid over the canvas, so they showed straight through surfaces that should have hidden them. We now test each label's line of sight against the surface's triangles.
- Zooming a long way. A straight zoom of 1000 times looks like a jump cut. We used the path of van Wijk and Nuij, which pulls back before it travels and keeps the destination in view on the way down. Then arithmetic became the limit: the graphics card computes \(z \to z^2 + c\) with about seven significant digits, so a fractal breaks up past roughly 5000 times magnification.
- A ball that bounces forever. A ball that keeps 80% of its speed bounces infinitely often, yet dropped from 2 m it comes to rest in a finite time. If the first fall takes \(T\), the flights add up to \(T + 2T(0.8 + 0.8^2 + 0.8^3 + \cdots) = 9T\), about 5.75 seconds. The simulation has to find each impact inside a time step and then let the ball rest, instead of chasing smaller and smaller bounces.
- Speech is not text.
x^2on screen has to be "x squared" out loud, Play has to wait for the sentence to end, and a spoken question has to pause the lesson without losing its place.
Accomplishments that we're proud of
- The physics engine is tested against known physics: a spring against its formula, a pendulum's energy and true period, the bouncing ball against its geometric series, and the energy of two chaotic double pendulums.
- Two olympiad problems (IMO 1959 Problem 5 and IMO 2011 Problem 2, the windmill) have hand-built solutions whose geometry is tested numerically. The app marks them as verified.
- A fluid simulation that runs live in the browser, inside an explanation, and responds to sliders while it runs.
- An interface in black and white, where the only colour on screen belongs to the mathematics.
What we learned
- Asking a model for data instead of code is what made everything else possible. Streaming, validation, editing and sharing all follow from it.
- A good explanation is mostly about attention. Deciding where the camera looks and what the pointer touches mattered more than adding kinds of graph.
- Numbers have edges. 32-bit floats end a fractal zoom, and the small-angle formula is 7.3% off at 60°. Showing those edges turned out to be good teaching material.
- A generated explanation is only as correct as the model's solution. The validator catches broken expressions, not wrong mathematics, which is why the verified problems exist and why checking generated solutions is the next thing to build.
- Two people can change one engine at speed if the tests check the mathematics. We both pushed to
mainall day, and the test suite is what kept it standing.
What's next for VisuMath
- Checking generated solutions numerically before they are shown, the way the verified problems are checked now.
- More physics: rigid bodies, and field lines in three dimensions.
- Putting it in front of students and teachers, and learning which steps they stop and ask about.
Built With
- claude-api
- elevenlabs
- glsl
- katex
- mathjs
- motion
- next.js
- playwright
- react
- react-three-fiber
- tailwind-css
- three.js
- typescript
- vitest
- web-speech-api
- webgl
- zod
- zustand