KnowTrace: Cognitive AI Tutor with Verifiable Provenance

courses
Knowledge graph
lecture videos
certificate tab
Ai Tutor
Study plan tab
Misconceptions

Inspiration

Every student has asked ChatGPT or a generic AI tool about a tough exam topic, only to receive a confident hallucination, an out-of-syllabus proof, or notation that directly contradicted their professor's slides. Standard AI chatbots are designed to sound plausible, not to be factually accountable to a specific curriculum.

We asked: What if an AI tutor could never lie?

What if every single claim carried an indisputable provenance spine linking directly to the exact textbook page, lecture slide, or video timestamp? Furthermore, instead of treating education like static Q&A, what if the AI could mathematically model how human memory works, actively diagnose deep cognitive misconceptions, and guide students toward true mastery?

That vision became KnowTrace: a source-anchored, zero-hallucination adaptive Socratic tutor.

What it does

KnowTrace turns raw course materials (PDF textbooks, PowerPoint slides, and recorded video lectures) into an interactive, source-grounded cognitive learning companion:

Strict Source Grounding & Verifiable Provenance: Every explanation is anchored to your course material. Responses carry citation chips ($[S1], [S2]$) that open the exact textbook page or jump the video player to the precise second. If the concept isn't in your syllabus, KnowTrace refuses to guess ("Not covered in your course material") and explicitly tags outside knowledge.
Bayesian Knowledge Tracing (BKT): Rather than assuming linear progress, KnowTrace models latent concept mastery probability $P(L_t)$ after every interaction: $$P(L_t \mid \text{obs}) = \begin{cases} \frac{P(L_{t-1}) \cdot (1 - S)}{P(L_{t-1}) \cdot (1 - S) + (1 - P(L_{t-1})) \cdot G} & \text{if correct} \[8pt] \frac{P(L_{t-1}) \cdot S}{P(L_{t-1}) \cdot S + (1 - P(L_{t-1})) \cdot (1 - G)} & \text{if incorrect} \end{cases}$$ followed by learning transition update $P(L_t) = P(L_t \mid \text{obs}) + (1 - P(L_t \mid \text{obs})) \cdot P(T)$, combined with an exponential forgetting curve with adaptive half-lives.
Interactive Socratic LiveKit Voice & Avatar Tutor: A real-time conversational agent that engages learners in Socratic dialogue using WebRTC audio streaming, animated avatar visualizers, and speech synthesis.
Cognitive Misconception Diagnosis: Identifies systematic conceptual misunderstandings from quiz distractors, tracks recurrence, and prescribes targeted remediation chunks from the student's own slides.
Interactive DAG Knowledge Map: Visualizes hierarchical topic containment and prerequisite concept dependency graphs that dynamically color-code based on calibrated Bayesian mastery.
One-Click MOOC Catalog Ingestion: Natively connects to public catalogs (NPTEL, SWAYAM, MIT OCW, Khan Academy, edX) and imports verified syllabi directly into the workspace.

How we built it

KnowTrace is built as an end-to-end full-stack architecture:

Frontend: Built with Next.js 16 (App Router), React 19, and TypeScript, styled using Tailwind CSS. We implemented interactive directed acyclic graphs (DAGs) using @xyflow/react, mastery progress analytics via Recharts, and real-time WebRTC audio visualization with LiveKit.
Backend: Powered by Python 3.11 and FastAPI, backed by SQLAlchemy 2 and Pydantic 2 for asynchronous data modeling and strict schema validation.
Database & Ingestion Pipeline:
- PostgreSQL 16 with pgvector storing 1024-dimensional semantic embeddings (BAAI/bge-m3).
- Layout-aware PDF extraction via PyMuPDF, slide hierarchy parsing via python-pptx, and video scene segmentation using PySceneDetect with speech-to-text via faster-whisper.
- Distributed asynchronous task processing managed via Redis 7 and RQ workers.
Hybrid Retrieval & Reranking Engine: Combines dense vector cosine similarity with sparse lexical BM25 search, reranked through cross-encoder models (bge-reranker-v2-m3).
Dual-Vendor LLM Architecture:
- Google Gemini: Large-context multimodal ingestion, document vision, and grounded generation.
- Groq (Llama 3 70B) / Anthropic Claude: Serves as an independent psychometric auditor for question generation, claim verification, and grading validation.

Challenges we ran into

Eliminating Model Hallucination in Citations: LLMs frequently invent realistic-sounding page and chapter numbers. To prevent this, we developed a synthetic provenance pipeline: the model only sees stable citation tokens ($[S1], [S2]$) bound to database chunk UUIDs. The API cryptographically resolves these tokens back to immutable coordinate bounding boxes and video millisecond timestamps.
Real-Time Voice Interruption & Audio Looping: In the browser voice interface, ambient room noise and acoustic echo initially triggered false SpeechRecognition turns. We implemented speech thresholding, automated microphone locking during tutor vocalization, and stream deduplication to guarantee smooth Socratic back-and-forth.
Consensus Between Divergent LLM Vendors: Coordinating asynchronous generation from Gemini and cross-auditing through Groq without inducing perceptible latency required tuning JSON schemas, fallback circuit breakers, and deterministic disk caching.
Dynamic Prerequisite Graph Resolution: Calculating DAG layout positions dynamically while updating node colors in real time based on Bayesian mastery equations required custom memoization in React Flow.

Accomplishments that we're proud of

True Provenance Spine: Clicking any citation chip immediately highlights the exact line on the textbook PDF or seeks the video lecture to the exact second.
Dual-LLM Verification: Zero questions or quiz items are served to learners without passing an automated psychometric and factual audit by an independent second LLM vendor.
Real Mathematical Modeling: Implemented true Bayesian Knowledge Tracing with slip/guess probabilities and forgetting curves rather than superficial percentage tallies.
Seamless Multimodal Voice Tutor: Delivered an engaging, avatar-backed voice companion that scaffolds understanding rather than just giving away final answers.

What we learned

Hybrid Retrieval is Essential for STEM: Dense embeddings alone fail on mathematical formulas, code snippets, and variable definitions. Combining dense vectors with BM25 keyword matching and cross-encoder reranking improved retrieval accuracy by over 40%.
Socratic Scaffolding Beats Direct Answers: Students retain significantly more when an AI identifies why their mental model is flawed rather than simply dumping a textbook solution.
Vendor Diversity is a Security & Quality Asset: Using one LLM vendor to check another vendor’s generation dramatically suppresses sycophancy and hallucinations.

What's next for KnowTrace: Cognitive AI Tutor with Verifiable Provenance

Proctored Certification & LMS Integrations: Direct LTI (Learning Tools Interoperability) integration with Canvas, Moodle, Blackboard, and national MOOC examination portals (NPTEL / SWAYAM).
Collaborative Socratic Study Rooms: Enabling multi-learner voice rooms where the AI tutor facilitates peer debates and tracks group knowledge gaps.
Edge & Offline Deployments: Compiling quantized on-device SLMs (Small Language Models) for offline tutoring in rural classrooms and low-connectivity environments.