Inspiration
Grade 12 is a year that defines a lot in South Africa. Your matric results follow you. I remember working through past papers a lot, they were some of the most useful thing I had. But what struck me was that students in the same grade, writing the same exam, were having completely different experiences depending on which class they landed in.
A student with a great maths teacher had something the others didn't. A student whose parents could afford a private tutor at R300 an hour had something else entirely. The ones who couldn't? Googling, watching YouTube, hoping to stumble onto the right explanation before the test.
Circle geometry clicked for me not from a textbook but because a teacher walked through a diagram with me, pointed at the angle at the centre, the angle at the circumference, showed me why one is twice the other using a drawing right in front of me. That is what tutoring actually is. Not a content library. Someone who watches you work and responds to your specific mistake.
Most AI chat apps break the moment you need to show your working. You draw on paper, photograph it, upload it, wait. The tutor feel is gone. And for students whose schools don't run proper chemistry or physics practicals, schools that can't maintain equipment, that don't have the budget, there's a whole category of exam question they go into blind.
Lyrn exists to close both gaps. The tutoring gap and the lab gap. The idea that where you were born or what school your parents could afford should determine how well you understand electrostatics or titration curves is something that is genuinely worth trying to fix.
What it does
Most AI tutoring tools respond to what you type. Mira responds to what you draw.
At the centre of Lyrn is Mira — a Socratic AI tutor with four modes of working with a student: live whiteboard, chat with inline visual diagrams, voice, and autonomous lab operation.
The whiteboard
The shared canvas uses Excalidraw. When a student sketches a circle theorem, a force body diagram, or the working for a quadratic, Mira reads it using real computer vision — the canvas is exported to PNG and sent to Claude's vision API, not described in text. Mira sees the actual drawing. She can write back on the board, annotate the student's diagram, add labels, draw new figures. A student who draws the wrong angle relationship sees Mira annotate directly on their figure showing where the reasoning broke.
When Mira is teaching on the whiteboard, her board writes are timed to her speech. If she is explaining that the angle at the centre is twice the angle at the circumference, the step she is writing appears on the board at the moment she says it — not before (which gives away the answer) and not after (which breaks the flow). It works the same way a person at a whiteboard actually teaches.
If Mira is mid-sentence and the student has figured it out, they can interrupt. She stops speaking. The student records their explanation. Mira listens and responds to what was actually said, not what she was about to say.
Chat with inline visual diagrams
For theory subjects — Accounting, History, Geography, Computer Science — Mira teaches through the chat interface and embeds live visual diagrams directly inline in the conversation. These are not static images: they are rendered SVG and HTML diagrams, generated by the model and drawn directly inside the chat using fenced diagram blocks. A lesson on double-entry bookkeeping produces an actual T-account diagram. A lesson on the causes of World War One produces an annotated timeline. A lesson on data structures produces a visual tree or linked list rendered in real time as Mira explains it. Students see the concept and the visual at the same moment they read the explanation.
The hidden prompt design means students never see the instruction Mira is given to start a lesson — they see only Mira beginning to teach, naturally, as if she chose to explain it that way herself.
Live lesson teaching
When a student starts a lesson on factorising or the theorem of Pythagoras, Mira does not hand them an explanation. She teaches. She plans a worked example, writes each step on the board while speaking it aloud, then poses a practice problem and waits for the student to attempt it. She reads their working off the board using vision, confirms it or identifies exactly where the reasoning broke down, and either moves forward or reteaches. Voice synthesis makes her sound like a person — not a TTS robot — and the interrupt/resume flow means the student can take over at any point.
Physics and chemistry labs
Students who attend schools without functioning science labs can describe an experiment to Mira. She will launch a real PhET physics simulation — a browser-based simulation used by universities worldwide — and operate it autonomously on the student's behalf. The agent builds circuits, configures wave interference setups, and adjusts parameters while narrating what it is doing in real time.
For chemistry, there is a full titration laboratory: a student adds sodium hydroxide to hydrochloric acid drop by drop, watches a live equivalence curve computed from real stoichiometric equations, and has Mira explain what is happening at each inflection point.
The circuit lab runs a physics solver built from scratch: Modified Nodal Analysis with Gaussian elimination. It stamps conductance into a matrix, solves for node voltages and branch currents, handles LDR resistance as a function of light level, blocks current through a reverse-biased LED, detects short circuits, and excludes dangling components from the solve. Parallel circuits carry different currents in different branches because the physics says they should, not because of a lookup table.
Adaptive tests and cognitive graph
Tests are generated per grade, per subject, across all eight subjects in the CAPS and IEB curricula: Mathematics, Physical Sciences, Life Sciences, English, Accounting, History, Geography, and Computer Science. Results update a cognitive graph — a force-directed node map, with real repulsion and attraction physics in the layout engine — that shows mastery per concept, colour-coded by subject, updated from actual quiz and lesson data. A student can see which nodes are solid and which are gaps before an exam.
Everything else
A command palette gives keyboard access to every feature. A learning paths view organises lessons by subject and grade. A past papers workspace lets students work through official exam papers with Mira available to help at any step. Grade selection updates the entire experience — lesson content, test difficulty, and curriculum alignment all shift when a student changes their grade from Grade 9 to Grade 12.
How we built it
Frontend
The application is a single-page app with no frontend framework — intentional, not a gap. A real-time tutoring experience coordinates live audio, streaming AI text, timed board writes, SSE consumers, and animated graph layouts simultaneously. Explicit control over the event loop makes that coordination cleaner than working against a reconciler. The codebase is 3,600 lines of JavaScript with 249 named functions and a 5,300-line CSS design system covering nine distinct views.
The entry bundle is 372KB gzipped. Clerk's 2.8MB authentication SDK is dynamically imported only when authentication is needed. The PhET lab module is a separate chunk loaded on demand. This matters for students on mobile data.
Backend and AI gateway
The Node.js server exposes a multi-provider AI gateway. Requests fall through from AWS Bedrock (primary) to the direct Anthropic API to Groq as a final fallback — in the same order for both streaming and non-streaming callers. If a lesson is mid-flow and the primary provider has a latency spike, the gateway recovers without the student seeing it.
Supabase stores all learning data — events, sessions, assessments, chat history — with row-level security enforced at the database level. Every table policy keys on the authenticated user's ID. Clerk handles authentication.
The PhET lab agent
The lab agent is 1,600 lines of Python. It uses Browser-Use — an AI agent framework for web automation — to drive a Chromium browser with real keyboard and mouse events. In production it attaches to a Steel-managed cloud browser via Chrome DevTools Protocol (CDP): Steel maintains the browser session and the agent connects to it remotely. In local development it attaches to a locally-running Chromium instance instead, using the same CDP interface — the agent code is identical in both paths.
The cloud session runs on ECS Fargate, provisioned on demand when a student opens the lab. There is no idle capacity. The gate requires PHET_ECS_ENABLED=true in the environment and validates that the configured cluster name contains lyrnos before any RunTask call. Sessions are hard-capped at seven minutes with automatic cleanup — Steel sessions are closed, ECS tasks are stopped, and the student's lab instance is marked released.
Before touching the simulation canvas, the agent plans the complete build — what components are needed, in what order, where they connect. It identifies terminal positions from a screenshot using vision rather than hard-coded coordinates, because terminal positions shift with zoom level and component orientation. It verifies the completed build from a fresh screenshot before reporting success. Every action is logged so a failure is diagnosable.
The trust boundary
Board images are validated on the way in: PNG only, 3.75MB ceiling, 120-element snapshot cap, annotation type allowlist. Mira's reading of a board is treated as a candidate interpretation — a separate deterministic solver checks any equations independently. If they disagree, Mira acknowledges uncertainty rather than teaching confidently from a wrong premise. This prevents a class of hallucination that would be genuinely harmful in an educational context: a student writes a wrong answer, the model misreads it as correct, and confidently explains why they got it right. That cannot happen in Lyrn.
The trust boundary is covered by 15 unit tests that run on every change.
Codex collaboration
We used Codex throughout, switching between Luna and Terra based on what each task actually needed. Luna for incremental edits, CSS iteration, and UI fixes where speed mattered more than deep cross-file context. Terra for the multi-file architectural passes — the voice state machine, the ECS safety gate, the trust boundary layer, the streaming gateway — where holding complex invariants across files in context mattered. Working within a credit budget forced the discipline to make that distinction deliberately rather than defaulting to the most capable model for everything.
Challenges we ran into
Timing board writes to speech was the first hard problem. The lesson plan comes back as a sequence of steps with associated board actions. Each written element must appear at the moment Mira finishes saying the corresponding sentence — the implementation tracks audio boundary events from the voice pipeline and sequences DOM writes against them.
The whiteboard OCR trust problem is fundamental: you want the model to be responsive to what a student draws, but a model that confidently misreads a student's working and teaches from the wrong premise is worse than no model at all. Treating the vision output as a candidate to be verified — rather than a fact to be acted on — was the right design, but it took careful work to implement without making Mira feel hesitant about clear cases.
The PhET agent's terminal identification was genuinely hard. Component terminals in the PhET circuit builder are at positions that depend on zoom, component orientation, and canvas layout. The only reliable solution was vision-based identification from the rendered screenshot — which required building a verification step to catch failures before they are reported as success.
Steel CDP session lifecycle across ECS Fargate required careful engineering. An ECS task that starts, fails to attach to Steel, and runs until the 7-minute cap would waste compute and block the student's next session. The agent's attach sequence has explicit failure detection and the hard cap enforces cleanup regardless.
Accomplishments that we're proud of
The circuit physics is correct. The MNA solver computes what the laws of physics say — a parallel circuit carries different currents in different branches, a short circuit is detected because the source current exceeds the physical limit, an LDR in a low-light environment has a different resistance than in bright light. No lookup table, no approximation.
The lesson teaching experience actually feeling like a tutor. A student working through factorising hears Mira speak and watches each step appear on the board at the moment she says it. That timing is not cosmetic — it is why the experience feels like being taught rather than reading a transcript.
The chat diagram rendering putting a T-account or a circuit schematic directly in the conversation, generated on demand, at the exact moment the explanation arrives. No screenshot. No static image. A live visual drawn by the AI as part of its response.
The bundle being 372KB for an application that does real-time voice, streaming AI, live board coordination, force-directed graph layout, and circuit physics simulation. Every kilobyte matters when a student is on a limited mobile data plan.
Building something genuinely useful to a student whose school has no functioning physics lab, who is writing matric in three months, who cannot afford a tutor — and giving that student the same quality of worked-example, hands-on, voice-guided, diagram-rich experience that a well-resourced student gets from a private tutor.
What we learned
The most useful thing Codex changed was how early architectural problems became visible. Working alone you can tell yourself a design is fine and continue. When you hand the task to a model and read what it produces, you see immediately whether your mental model of the system is actually coherent. The trust boundary layer is the clearest example — having to articulate precisely why board readings cannot be trusted as ground truth produced a cleaner implementation than solo reasoning would have.
Luna versus Terra is a real distinction, not just a cost optimisation. The question "do I need iteration speed or cross-file coherence right now" turns out to be a useful frame for any kind of engineering decision, AI-assisted or not.
Designing for a learner's cognitive state rather than engineering convenience. The interrupt flow, the step-timed board writes, the practice question before moving on — none of that comes from what is easiest to build. It comes from thinking about what it feels like to be mid-explanation and not quite understand yet. Getting that right required resisting the easier version at every step.
What's next for LyrnOS
South Africa first, then other national curricula. The vision tutor, lesson engine, lab agent, and subject database all work independently of which curriculum they serve. CAPS and IEB are the beachhead because that is where the inequity is most immediate and where the product is most directly tested.
Exam preparation mode: connecting the cognitive graph to past paper analysis so Mira can look at a student's actual mastery data, identify the concepts with the most remaining gaps given how many weeks are left before the exam, and build a preparation schedule around real weaknesses rather than a generic syllabus order.
The PhET simulation library has hundreds of experiments. The agent supports a working set today. Building the guided experiment flows that turn each simulation into a structured lesson — with Mira present throughout, asking questions and testing understanding, is the work that makes the lab a full curriculum track rather than a demonstration.
And the classroom version. A teacher who can see which concepts her students are collectively struggling with, in aggregate, in real time, based on what Mira has tried to explain and where students got stuck, changes what is possible in a classroom. That version of Lyrn is worth building.
Log in or sign up for Devpost to join the conversation.