The problem

STEM students — especially secondary and early college learners in Pakistan — do not fail for lack of videos. They fail when a misconception stays invisible. A static YouTube clip cannot see that a student wrote (x^2+9) instead of ((x+3)^2). A chatbot that only returns paragraphs cannot draw the missing middle term on a board, check understanding, or remember the lesson the next time they log in.

Students need a teacher workflow: see their mistake, explain it step by step on a whiteboard, ask questions mid-lesson, practice, and keep a durable record under a real account.

Who it’s for

Secondary and early college STEM students (including NED undergraduates) who get stuck when classroom or video explanations move too fast — algebra first, then any short topic they paste in.

What we built

Visual AI Tutor is a deployed web app where AI is the core of the lesson, not a bolted-on chat box.

  1. Sign up + TOTP 2FA (Supabase Auth) — isolated workspaces per student.
  2. Open topic: paste a short brief (e.g. photosynthesis) → Gemini returns Zod-validated board steps → live draw on an SVG whiteboard (optional Excalidraw) with captions and voice.
  3. Curriculum pack: Binomial Squares → submit a wrong expansion → Misconception Detective diagnoses → Play teaches the area model → board Q&A → select both 3x regions (graded in app code) → quiz → lesson summary.
  4. Stretch on the same URL: live viva and Teach-the-AI (AI plays a confused learner; the student teaches back).
  5. Persistence: lessons survive sign-out and restart in Supabase Postgres with RLS. A second account cannot see another student’s data.

Captions stay on even if audio fails. Voice uses Vapi when configured, otherwise Web Speech. We never draw unvalidated board JSON and never present hardcoded text as a live model call. Pack algebra and the 2bx board task are graded in application code (mathjs + templates); Gemini explains after the app scores.

How AI is used (M1)

Live calls to Google Gemini power:

  • misconception diagnosis (Detective)
  • board tutoring (step operations + spoken lines)
  • mid-lesson board Q&A
  • quiz coaching (after app grades)
  • lesson summary
  • viva questions + rubric scoring
  • Teach-the-AI plant + score

Every structured model response is Zod-validated before it is drawn or shown. Judges can trigger these features on the live URL and find the API routes in the public repo.

Tech stack

  • App: Next.js (App Router), TypeScript, Tailwind, shadcn/ui
  • Auth + data: Supabase Auth (email + TOTP MFA), Postgres, Row Level Security
  • AI: Google Gemini (gemini-3.5-flash-lite by default), Zod, mathjs
  • Board: SVG default; optional @excalidraw/excalidraw
  • Voice: Vapi (optional) → Web Speech fallback; captions always on
  • Host: Vercel

DevMarketplace gigs (M6)

  1. Supabase TOTP MFA gate helpers (Next.js)
  2. Zod board ops + narration steps (whiteboard tutor)

Built With

  • excalidraw
  • google-gemini
  • mathjs
  • next.js
  • postgresql
  • shadcn/ui
  • supabase
  • tailwind-css
  • typescript
  • vapi
  • vercel
  • web-speech-api
  • zod
Share this project:

Updates

Submission history