AI TUTOR CEINTELLY

✨ What It Does

AI TUTOR CEINTELLY is an end-to-end autonomous agentic study suite equipped with 7 core studio engines, cloud database persistence, and real-time observability:

🧠 Student Knowledge Graph & Mastery Engine: Autonomously tracks concept mastery scores ($M \in [0.0, 1.0]$) across Bloom's Cognitive Taxonomy. It isolates recurring knowledge gaps and dynamically recommends targeted remedial drills.
📐 Interactive Audio-Visual Whiteboard Walkthroughs: Synthesizes synchronized vector geometry, free-body diagrams, tension vectors, and mathematical derivations on an $800 \times 500$ coordinate canvas with synchronized spoken voiceover and an interactive student stylus.
🎙️ NotebookLM-Style Dual-Host Audio Podcast Studio: Absorbs conversation history and generates an authentic, two-speaker spoken academic discussion between hosts Alex & Sam, complete with live word-level autoscrolling transcripts and show notes.
⏱️ Timed Mock Exam & Socratic Proctoring Simulator: Replicates high-stakes standardized exams (MCAT, USMLE, AP Physics) with real-time timers, tab-switch proctoring diagnostics, mastery tier scorecards, and 1-click remedial study set generation.
📝 Live Scratchpad & Proactive Socratic Scaffolding: Operates as a background listener during study sessions, autonomously extracting structured Markdown notes, LaTeX formulas, and proactive hints without interrupting learner flow.
📄 Multimodal Document & Textbook Ingestion Engine with pgvector RAG: Deep-parses 50+ page PDFs, lecture slides, and notes using Gemini's 1M token context window. Ingests semantic chunks and 768-dimensional embeddings (text-embedding-004) directly into Google Cloud SQL for fast cosine distance similarity retrieval ($\text{dist} = 1 - \frac{\vec{u} \cdot \vec{v}}{|\vec{u}|_2 |\vec{v}|_2}$), plus downloadable Microsoft Word (.docx) study packs.
📅 Taskmaster Curriculum & Syllabus Studio: Asynchronously plans personalized multi-module learning roadmaps with hourly pacing milestones and active recall checkpoints.
⚡ Gemini 3.7 Live Voice & Vision: Bidirectional real-time voice and vision with camera frame analysis and customizable Socratic pedagogy personas.
⏱️ Pomodoro Focus Hub & Generative Soundscapes: Gamified study timer featuring binaural beats, ambient generative audio, streak trackers, and unlockable achievement badges.
📊 Cloud SQL Health & Telemetry Dashboard: Real-time connection status monitoring, sub-10ms query ping latency metrics, pgvector extension verification, and record volume counters inside the settings panel.

🛠️ How We Built It

Core AI Intelligence: Google Gemini 3.7 Flash (gemini-3.7-flash) via the official @google/genai TypeScript SDK, utilizing its $1,048,576$-token context window for full textbook ingestion and $65,536$-token output ceiling.
Vector Embedding Engine: Google GenAI text-embedding-004 generating 768-dimensional dense vectors for semantic document chunking and retrieval.
Cloud Database & Persistent Storage: Google Cloud SQL (PostgreSQL 16) instance (ai-studio-d0a96dea) in asia-southeast1 equipped with the native pgvector extension and Drizzle ORM type-safe schema mapping (study_embeddings, study_records, users).
Cloud & Compute Infrastructure: Containerized Node.js/Express backend deployed on Google Cloud Run with automated ingress routing, sub-second container cold starts, and scale-to-zero operational efficiency.
Frontend & Visual Engine: React 18, TypeScript, Vite, Tailwind CSS, Lucide Icons, and Framer Motion layout transitions. Interactive coordinate diagrams are rendered via a custom HTML5 Vector Canvas engine.
Offline-First Storage: IndexedDB (aitutor_study_vault_db) for zero-latency local caching of study sets, transcripts, diagnostic reports, and student mastery vectors.
Document & Speech Engines: Web Speech API for bidirectional voice/podcast synthesis and a client-side compilation pipeline producing formatted Microsoft Word (.docx) documents.
Security & Privacy (BYOK): Salted key derivation cipher with HMAC integrity verification (enc:v1:<salt>:<iv>:<cipher>:<mac>), ensuring API credentials are never stored in plaintext.

🧗 Challenges We Ran Into

Dynamic Vector Blackboard Geometry: Generating reliable mathematical vector diagrams (e.g., force vectors $\vec{F}_{\text{net}} = \sum \vec{F}_i = m\vec{a}$, coordinate transformations, and geometric angles) required engineering an auto-scaling $800 \times 500$ virtual coordinate system with collision-free labeling.
Dual-Speaker Audio Synchronization: Orchestrating turn-taking between two distinct vocal profiles (Alex & Sam) required precise millisecond-level timestamp alignment and continuous autoscroll tracking.
High-Performance pgvector Cosine Search in Cloud SQL: Structuring vector queries with raw SQL bindings in Drizzle ORM (ORDER BY embedding <=> $1 ASC LIMIT $2) while seamlessly managing connection pools across container cold starts.
Universal Context Absorption: Implementing the ConversationSourceSelector across 7 distinct studio tools required building a decoupled context extractor (extractConversationStudyContext) that processes multi-turn messages and image attachments without UI freeze.
Pedagogical Balance: Tuning system prompts to ensure the AI acts as a genuine Socratic mentor—guiding students with progressive hints rather than simply giving away direct answers.

🏆 Accomplishments That We're Proud Of

🥇 True Autonomous Multi-Agent Suite: The agent actively takes notes, diagnoses misconceptions, computes semantic embeddings, and builds study assets in the background without needing manual prompting.
🎨 Multimodal UX Synergy: Seamlessly bridges text chat, live voice/vision, interactive chalkboard diagrams, spoken dual-host podcasts, proctored exam countdowns, and real-time database telemetry into one unified, accessible interface.
⚡ Production-Ready Google Cloud Stack: Full serverless deployment on Google Cloud Run linked to a managed Google Cloud SQL instance with active pgvector cosine distance indexing.
📱 Mobile & Desktop Responsiveness: Engineered responsive drawers, collapsible studio toolbars, and strict modal isolation that function smoothly across smartphones, tablets, and desktops.
🔒 Zero-Trust Encryption: Robust client-side salted cipher encryption keeping student API keys secure.

📚 What We Learned

Socratic Scaffolding Yields Deeper Retention: Guided inquiry and step-by-step diagnostic verification lead to significantly better concept mastery than passive answer generation.
Multimodal Visual Grounding: Pairing spoken voiceover with synchronized chalkboard vector graphics drastically lowers cognitive load for complex STEM topics.
Grounded Vector Retrieval (RAG): Combining Gemini 3.7 Flash's 1M context with Cloud SQL pgvector similarity search provides instant, hallucination-free citations from uploaded textbooks.
Background Agent Execution: Moving heavy-lifting tasks (note taking, diagnostic scoring, syllabus generation) into background agents allows students to stay in deep flow.

🚀 What's Next for AI TUTOR CEINTELLY

Real-Time Multiplayer Socratic Study Rooms: Collaborative peer study sessions with synchronized whiteboards and group proctored exams over WebRTC.
Automated Spaced-Repetition Schedule: Automated notifications and flashcard review intervals scheduled via Cloud Scheduler and Cloud Pub/Sub.
Anki (.apkg) Mobile Package Export: Direct binary generation of spaced-repetition mobile decks for on-the-go review.