Kognit

Stop copy-pasting answers. Kognit listens as you code out loud, using Socratic guidance and real-time visual feedback to help you master debugging.

Comment

Kognit — Real-Time Socratic AI Coding Tutor

Inspiration

In the world of developers, things have gotten a little weird. Tools like ChatGPT and GitHub Copilot can generate code incredibly fast. But while we can write code faster, we do not always understand what the code is actually doing.

Many developers get stuck in a repetitive loop:

  • Copy and paste generated code.
  • Run it.
  • Encounter an error.
  • Paste the error back into an AI tool.
  • Repeat — without understanding why the code failed.

We started thinking about a different approach:

What if we had a coding tutor that behaved like a great professor — one that did not simply give us the answer, but helped us discover it ourselves?

One effective learning technique is the think-aloud protocol. While solving a problem, students verbalize what they are thinking. Their reasoning, hesitation, and uncertainty give a teacher insight into where their mental model is breaking down.

That idea became Kognit.

Kognit listens to your reasoning while observing the code you write. It can execute that code in an isolated environment, detect where you may be stuck, and respond with carefully designed Socratic hints.

It acts like an intelligent pair programmer — not one that does the work for you, but one that gives you the right push to solve it yourself.

What It Does

Kognit is a real-time Socratic AI coding tutor designed to transform cognitive friction into learning breakthroughs.

  • Voice-First Socratic Guidance: Listens as you think aloud while coding and evaluates speech transcripts alongside code changes. Using ElevenLabs neural audio, Kognit responds with short probing questions of fewer than 25 words.

  • Debounced Code & Intent Analysis: Observes Monaco Editor deltas in real time to identify logic loops, missing edge cases, off-by-one errors, and syntax mistakes before the student explicitly asks for help.

  • Emotion-Aware AI Character: A vector-based Kognit avatar uses dual-jointed skeletal arm rigging and expressions such as happy, focused, panic, and shock to reflect the student's cognitive state.

  • Neural Synapse Visualizer: A 60 FPS canvas visualization dynamically changes particle motion, rotation, jitter, and visual intensity based on calculated confusion metrics.

  • Dynamic Skill Tree & Knowledge Graph: Maps error patterns such as recursion_base_case, null_reference, and off_by_one into a persistent mastery engine that tracks conceptual progress over time.

  • Isolated Sandbox Execution: Executes Python, JavaScript, C++, Java, Rust, and Go through the Piston API, feeding stdout and stderr directly into the Socratic reasoning pipeline.

How We Built It

Kognit combines an event-driven gateway, multi-LLM orchestration pipeline, isolated sandbox engine, and dynamic Next.js 16 + React 19 interface.

1. Real-Time Voice & Event Gateway

  • Audio + Code Streaming: PCM16 microphone audio and debounced Monaco code changes are multiplexed through a single WebSocket connection.

  • Speech Pipeline: Silero VAD detects utterance boundaries, while Groq Whisper performs low-latency speech-to-text transcription.

  • Voice + Barge-In: ElevenLabs streams tutor speech back to the client. A barge_in event immediately terminates playback when the student begins speaking.

2. Multi-LLM Socratic Engine

Confusion Detection

Transcript and code context are jointly analyzed to generate a normalized confusion score:

$$ C \in [0,1] $$

When

$$ C < 0.35 $$

LLM generation is bypassed, reducing unnecessary API calls and response latency.

Socratic Questioner

Kognit generates one short probing question without revealing code or explicitly identifying the root cause.

When the escalation threshold reaches

$$ \text{level} \geq 4 $$

the system switches to a bounded direct hint to prevent learner frustration.

Multi-LLM Fallback Routing

Google Gemini acts as the primary reasoning model, while Groq Llama 3 provides automatic fallback during latency spikes or rate limits.

Mastery & Spaced Repetition

Concept mastery is persisted in Neon PostgreSQL using an adaptive ease factor:

$$ E_n = \max\left(1.3, E_{n-1}+\Delta\right) $$

where

$$ \Delta = \begin{cases} +0.1, & \text{if resolved quickly} \[4pt] -0.15, & \text{otherwise} \end{cases} $$

3. Adaptive UI & Infrastructure

  • Emotion-Aware Avatar: SVG skeletal joints dynamically represent the student's current cognitive state.

  • Act-Synapse Visualizer: Neural nodes smoothly interpolate toward real-time confusion targets using:

$$ P_{\text{current}} = P_{\text{current}} + \left(P_{\text{target}}-P_{\text{current}}\right) \cdot \min\left(1,\Delta t\cdot k\right) $$

  • Backend Infrastructure: Built with FastAPI, Neon PostgreSQL, SQLAlchemy/asyncpg, and Upstash Redis for persistence, asynchronous database access, session state, caching, and pub/sub.

Challenges We Ran Into

1. Audio Barge-In

Challenge: Student speech could overlap with active AI voice responses.

Solution: Instant barge_in events cancel ElevenLabs playback, flush client audio queues, and release backend session locks.

2. Multi-LLM Latency

Challenge: Model latency and rate limits affected response consistency.

Solution: An asynchronous fallback router uses Gemini as the primary model and Groq Llama 3 as fallback, supported by Redis caching.

3. Socratic Constraint Enforcement

Challenge: LLMs naturally tend to reveal solutions or generate code.

Solution: JSON schema constraints, few-shot prompting, and regex validation enforce short, question-based guidance without leaking solutions.

4. Avatar Stabilization

Challenge: Rapid expression changes initially caused vector joints to detach.

Solution: We rebuilt the arms as hierarchical skeletal chains anchored to fixed shoulder and elbow pivots:

$$ (cx,cy)_{\text{shoulder}}=(78,174) $$

$$ (cx,cy)_{\text{elbow}}=(60,210) $$

Accomplishments That We're Proud Of

  • Sub-500ms Voice Loop: Achieved a near-conversational pipeline combining Groq Whisper STT, Gemini Flash reasoning, and ElevenLabs audio streaming.

  • Zero Code-Leak Reliability: Built strong pedagogical guardrails that maintain Socratic questioning across extensive edge-case testing without directly exposing code solutions.

  • Fluid 60 FPS Reactive UI: Decoupled LLM inference from frontend rendering, allowing the synapse visualizer and Kognit avatar to remain smooth using Zustand and exponential interpolation.

  • Unified Full-Stack Architecture: Integrated Next.js 16, React 19, FastAPI, Piston, Clerk, Neon PostgreSQL, and Upstash Redis into a single real-time learning platform.

What We Learned

  • Multi-Modal Context Beats Code-Only Analysis: Combining speech transcripts with real-time code edits provides richer signals for detecting gaps between a student's intention and implementation.

  • Pedagogical Guardrails Improve Learning: Restricting responses to a single Socratic question of fewer than 25 words reduces cognitive overload and encourages active problem-solving.

  • Decoupled Architecture Is Critical for Real-Time AI: Separating low-latency WebSocket events, AI inference, and frontend rendering helps maintain a responsive experience across Next.js, React, and FastAPI.

What's Next for Kognit

  • Multi-File & Full Repository Context: Extend the reasoning engine to understand complete repositories using AST analysis, semantic embeddings, and pgvector.

  • Mock Exam Arena: Expand timed coding challenges where Kognit supervises students in real time and evaluates solution complexity, alongside problem-solving speed and conceptual mastery.

  • 3D Cyber-Classroom: Expand office-scene.tsx into customizable study environments with interactive avatar skins and gamified learning experiences.

Kognit doesn't code for you. It teaches you how to think through code.

Built With

Share this project:

Updates