Vision Tutor

Private tutoring costs anywhere from £40 to £100 an hour. As a parent, I know that feeling all too well — your child is stuck on a maths problem at 8pm, you're exhausted from work, and you can't remember how long division works either. You want to help, but you just can't. And hiring a tutor for every homework struggle simply isn't realistic for most families. That frustration became the spark for VisionTutor. I wanted to build something that gives every child access to a patient, knowledgeable tutor — not one who just hands over the answers, but one that guides them to think, just like a great teacher would. And I wanted any parent to be able to hand their child a phone and say: "show it your homework."

VisionTutor is a real-time AI-powered homework tutor. A child points their phone or laptop camera at their homework, and an AI tutor sees the question, explains it verbally in a warm and natural voice, and guides the student to the answer using the Socratic method — asking questions rather than giving answers away.

In real time, VisionTutor: Sees the homework through the device camera using WebRTC Speaks back to the student in a natural human voice, explaining concepts and asking guiding questions Listens to the student's verbal responses and adapts its coaching Surfaces hint cards on screen — short, digestible tips extracted automatically from the tutor's explanation Tracks progress over time, logging session transcripts and subject streaks to a personal dashboard

It works on any device with a camera and microphone — no app download needed.

VisionTutor is built entirely on Google Cloud. Every layer of the stack — from the AI brain to the database to the deployment pipeline — runs on Google's infrastructure. Google Cloud services used:

Gemini 2.5 Flash Native Audio Live API — The core AI engine. We use the Gemini Live API (via the GenAI SDK, v1alpha) to establish a persistent, bi-directional WebSocket connection between our backend and the model. Gemini simultaneously receives the camera video frames and the student's microphone audio, processes them together, and streams spoken audio responses back in real time. This is what gives VisionTutor its natural, conversational feel. Gemini 2.5 Flash (text) — Used as a fast, lightweight second model running asynchronously in the background. Each time the tutor speaks, a separate call to Gemini Flash analyses the transcript and extracts a short "hint" or "tip" card for the student's screen — without slowing down the live conversation. Firebase Authentication — Handles secure email/password sign-up and login. Every WebSocket session is authenticated using a Firebase ID token, ensuring only verified users can connect to the tutor. Cloud Firestore — Our NoSQL database. Session transcripts, hint history, learning streaks, and subject progress are all written to Firestore at the end of each tutoring session, powering the student's progress dashboard. Google Cloud Run — The FastAPI Python backend runs as a serverless container on Cloud Run. It handles all WebSocket connections, proxies communication to the Gemini Live API, and manages session state. Cloud Build + Artifact Registry — Our CI/CD pipeline. Cloud Build compiles and containerises the backend, pushes Docker images to Artifact Registry, and triggers rolling deployments to Cloud Run automatically. Google Cloud Storage — Stores AI-generated educational diagrams (via Vertex AI Imagen 3) that the tutor can produce to help visual learners. Images are uploaded to GCS and served to the frontend as diagram cards. Secret Manager — All sensitive credentials — Gemini API keys, Firebase service account — are stored in Secret Manager and accessed securely at runtime in production, never hardcoded.

Frontend: React + Vite + Tailwind CSS, using the browser's native WebRTC (getUserMedia) API for camera and microphone capture, and the Web Audio API for PCM audio streaming at 16kHz mono.

The biggest challenge — by a long margin — was getting the Gemini Live API connection to actually work reliably. The Gemini Live API is brand new, the documentation is sparse in places, and there were a surprising number of subtle configuration traps. Audio format had to be exactly PCM at 16kHz mono. The API version had to be v1alpha — the standard endpoint didn't support the live models. The model name changed between regions. The Google GenAI SDK auto-detects GOOGLE_CLOUD_PROJECT and silently re-routes traffic to Vertex AI instead of AI Studio — which then broke the connection entirely because the live audio models weren't available there. And then there was the debugging ritual. Every time we made a change, we'd start a session and say "Hi" — just to see if the tutor would respond. Hundreds of "Hi"s. Waiting. Silence. Error 1011. Tweak something. Say "Hi" again. It became a running joke. When the tutor finally said "Hi" back for the first time in a natural, warm voice — it genuinely felt like a moment. Getting WebSocket bidirectional streaming right in an async Python environment was also non-trivial. Managing two concurrent async tasks (receiving from Gemini while simultaneously receiving from the client) required careful use of asyncio.wait(FIRST_COMPLETED) to avoid race conditions and dropped messages.

We're most proud that it actually works — a child can point a camera at a piece of homework and have a real, flowing conversation with an AI that can see what they're looking at, speak back naturally, and guide them without just giving the answer away. That loop — camera on, AI responds, child speaks, AI listens and adapts — is genuinely magical the first time you see it. We're also proud of the hint extraction pipeline. Rather than overwhelming the child with long explanations, the system silently runs a second AI call in the background to distil the tutor's words into a short, memorable tip card — all without interrupting the live conversation. It's a small detail that makes the experience feel polished.

We learned that building with cutting-edge AI APIs means accepting that the documentation will sometimes lag behind the actual API behaviour — and that patience, systematic debugging, and good logging are your best tools. We also learned how powerful Google's unified cloud ecosystem is when everything talks to each other. Having Firebase Auth, Firestore, Cloud Run, Secret Manager, and the Gemini API all in the same project — with shared IAM and credentials — removed enormous amounts of integration friction compared to stitching together services from different providers. And personally, building VisionTutor reinforced something I already believed as a parent: the most important thing a tutor can do is ask the right question at the right moment. Getting Gemini to do that consistently — and naturally — was harder than expected, and more rewarding when it worked.

What's Next for VisionTutor ? The core experience works, and that's exciting — but there's a long road to making this something every family can rely on day to day. The immediate priorities are personalisation and production hardening. On the personalisation side, we want VisionTutor to remember a student across sessions — their strengths, the subjects they struggle with, their preferred way of being explained to — and adapt its tutoring style accordingly. A child who learns visually should get more diagrams. A child who's been stuck on fractions for three weeks should get a different approach than one encountering them for the first time. On the production side, we want to add proper rate limiting, replace debug logging with structured observability, tighten security (CORS, secrets hygiene), and build out the progress dashboard into something a parent can glance at and immediately understand how their child is progressing. Longer term, we want to expand beyond homework help — into exam preparation, reading comprehension, and eventually subject-specific tutoring modes for STEM, languages, and creative writing. The goal is simple: make a great tutor available to every child, not just those whose parents can afford one.