Zanoba AI

Zanoba AI — Your Intelligent, Real-Time Personal Tutor.

Comment

Inspiration

Online tutoring today is one of two things. A human, who is good, and who is priced and scheduled like a human. Or a chatbot, which answers questions but does not teach. It never opens the worksheet, never notices you went quiet on question 3, never writes the working out while it explains.

We wanted the second thing to behave like the first. Not an assistant that waits to be asked, but a tutor that has read your file, planned your hour before you arrived, and starts the lesson when you join.

What it does

A student books an hour. Before they arrive, an agent decides what to teach them: what the syllabus says next, what they actually know, and which objectives fit sixty minutes. Then it writes the worksheet.

Then the student joins a call and is taught. Out loud. The tutor greets them, turns to the page it means, writes on the paper a character at a time while it explains, circles a wrong answer, fills in a gap, and records what it noticed for next time.

How we built it

Preparation is a graph. One Google ADK Workflow, on Gemini 3.5 Flash, using four patterns because the work needs four.

  • Sequential for the first three stages. The curriculum agent must name a target lesson before the diagnostic agent can judge readiness for that lesson. Running these in parallel would not be faster. It would be wrong.
  • A conditional branch to a language or a STEM planner.
  • A join, where both branches meet one quality checker, so quality is defined once.
  • A bounded loop, where failed material is rewritten and re-graded, at most three times.

Agents pass data through session state, never through the prompt. Each declares an output_key and the next reads it back. A missing key raises rather than quietly inventing an answer. An objective agent improvising objectives with no placement to work from looks perfectly plausible, which is exactly what makes it dangerous.

The lesson is not a graph. It is a single ADK agent with twelve tools on a bidirectional audio stream, running on Gemini 3.1 Flash Live. A lesson is one continuous conversation, and the branching belongs to the student.

The stack. Cloud Run runs the agent with session affinity, because a class lives in one instance's memory. Firestore holds the student profile, the lesson history and the prepared material cache. A Next.js front end carries the microphone up as raw PCM and streams the tutor's own voice into a rendered avatar for lip sync.

Mastery per concept is tracked, and the diagnostic agent routes on it. Below threshold, the lesson opens with prerequisite review instead of the new material.

Challenges we ran into

The tutor never spoke first. The socket opened, the queue sat empty, and the model waited for a student who was waiting for the model. Twenty seconds later a watchdog declared a failed connection. A spoken lesson that worked perfectly had never once made a sound.

Sample rates are not a detail. Gemini Live speaks at 24 kHz and the avatar listens at 16 kHz. Getting it wrong does not raise an error. It plays a third fast and a fifth high, and the lips agree with it perfectly.

Sending the avatar text was the wrong idea. Handing it a transcript meant waiting for the whole turn to finish, paying for a second text to speech engine, and getting a different voice. Gemini emits PCM and the avatar accepts PCM. Piping one into the other deleted an entire stage and made the tutor answer instantly.

Accomplishments that we're proud of

It teaches, and we can prove it. Not "responds about maths". It opens the worksheet, turns to the page it means, writes the line a character at a time while it says the sentence, circles the wrong answer and fills in the gap. Each of those is a tool call we watched land on a real page in a real browser.

The tutor's face speaks the tutor's own voice. Most avatar integrations hand a transcript to a text to speech engine and get a second voice, a second bill and a turn's worth of delay. Gemini Live emits PCM and the renderer accepts PCM, so we pipe one straight into the other. The mouth moves to the same audio the student hears, and the answer starts as soon as the model does.

Two full curricula, end to end. French Sixième mathematics, with 299 hand encoded items, and German A1. The preparation graph is subject agnostic, so adding a subject is curriculum work rather than agent work.

Nearly four hundred tests, none of which need the network. The whole pipeline can be exercised offline, for free, before a single Gemini call.

The failure modes are legible. A tutor that stops is obvious. A tutor that smiles and says nothing looks exactly like a tutor that is thinking. The system now tells the difference between "she is broken" and "there was nothing to say", in the interface and in the logs, because we lost hours to not being able to tell.

What we learned

Green tests are not a working feature. Every bug above passed type checking, linting and a suite of nearly four hundred tests. Not one was found by reading the code. They were found by opening a browser, connecting to the socket, and counting what actually came down it. The first honest measurement said one frame and three bytes, where a working lesson sends 44 frames and ten seconds of speech.

The other lesson is about failure modes. A tutor that stops is obvious. A tutor that smiles pleasantly and says nothing looks exactly like a tutor that is thinking. We ended up building the system so that it says which half is broken.

What's next for Zanoba AI

Per session state. The live tutor's tools are currently pointed at a lesson through module level globals, so one instance teaches one class. Cloud Run's session affinity hides this today, but before two students can share an instance it becomes a per session context object. It is the first thing we would fix, and it is written down in the README.

A tutor who moves. She lip syncs but does not gesture, because she is built from a single photograph. A filmed avatar, or a higher fidelity generator, and she gestures. No code changes.

A student model that decays. Mastery is a fraction $m \in [0,1]$ recorded per concept. It should fall with time and be revisited on a spaced schedule, so the diagnostic agent reopens a misconception three weeks later instead of trusting a number from a lesson the student has half forgotten.

More subjects. The graph routes on domain and does not care which. The work is encoding curricula and writing the material rules that go with them.

A report the parent reads. Every lesson already writes history: what was covered, what was missed, which mistakes recurred. That is a weekly summary waiting to be generated, and it is the thing a paying parent actually wants.

Cloud SQL. The web app runs Prisma on SQLite for the hackathon. It is a connection string away from managed Postgres.

Built With

Share this project:

Updates

Submission history