CounterTutor

What if the AI was the struggling student? CounterTutor makes intentional SAT mistakes, learns from your explanations, and adapts until you can catch, explain, and apply the concept yourself.

Comment

CounterTutor

Inspiration

Most AI tutors follow the same pattern:

AI asks → Student answers → AI grades → AI explains.

We wanted to reverse that relationship.

Students in underserved schools often have access to questions and answers, but not necessarily to someone who can listen to their reasoning and challenge their misconceptions.

A private tutor doesn't just tell a student whether an answer is correct. They ask:

"Why do you think that?"

That interaction is difficult to reproduce in traditional self-study tools.

So we built CounterTutor.

Instead of the AI playing the tutor, the AI plays the struggling student. It intentionally makes realistic SAT mistakes, and the human becomes the tutor. The student must:

CATCH → EXPLAIN → APPLY

The goal is not simply to get students to answer correctly. The goal is to make them reason.

What It Does

CounterTutor is an AI-powered learning agent for SAT Math.

The student is shown an AI student's attempted solution. The AI may:

  • make a calculation mistake
  • apply a mathematical rule incorrectly
  • make a subtle reasoning error
  • occasionally solve the problem correctly

The student must determine whether the AI made a mistake and explain why.

CounterTutor evaluates the student's explanation and maintains a live mastery state. Depending on the evidence, the agent can:

  • PROBE the student's reasoning
  • REPEAT the misconception in a new context
  • TRANSFER the concept to an unseen problem
  • RETIRE a mastered misconception
  • ADVANCE to another weak concept

The student therefore isn't simply answering questions. They are teaching the AI.

The Core Learning Loop

┌─────────┐
│  CATCH  │
└────┬────┘
     ↓
┌─────────┐
│ EXPLAIN │
└────┬────┘
     ↓
┌─────────┐
│  APPLY  │
└────┬────┘
     ↓
┌─────────┐
│ REMEMBER│
└────┬────┘
     ↓
┌─────────┐
│  ADAPT  │
└────┬────┘
     │
     └──────────→ NEXT LEARNING CHALLENGE
  • A student who catches an error but cannot explain it is not treated as having mastered the concept.
  • A student who can explain the mistake but fails to apply the concept to a new problem is still developing.
  • Only demonstrated transfer can produce high-confidence mastery.

How the Agent Works

The LLM is not responsible for deciding what the agent should do. It interprets the student's natural-language explanation and returns structured evidence. The deterministic agent engine then makes the actual decision. This distinction is intentional.

The Agent Loop

Student explanation
        ↓
LLM extracts evidence
        ↓
Deterministic state update
        ↓
Agent evaluates mastery
        ↓
    NEXT ACTION
        ↓
Probe / Repeat / Transfer / Retire

The core agent logic can therefore be tested without an LLM. The LLM handles language. The agent owns the state and decisions. This prevents CounterTutor from being simply an LLM with a clever system prompt.

Why the AI Makes Mistakes

If the AI were wrong every time, students could simply learn:

"The AI said it, therefore it must be wrong."

That would completely destroy the learning objective. CounterTutor therefore includes both incorrect and correct AI attempts. The student has to genuinely evaluate the reasoning rather than blindly correcting the AI.

The AI's mistakes are selected from a curated misconception bank rather than invented randomly. This lets the system target specific mathematical misconceptions such as:

  • negative distribution
  • inequality direction
  • exponent rules
  • percentage bases
  • slope interpretation
  • ratios
  • mean and median reasoning
  • radius vs. diameter
  • Pythagorean theorem errors

Mastery Model

CounterTutor does not treat one correct response as mastery. Evidence accumulates across the interaction. A simplified confidence model considers:

$$ C = 0.25 \cdot C_{\text{catch}} + 0.35 \cdot C_{\text{explain}} + 0.40 \cdot C_{\text{apply}} - P_{\text{false}} $$

where:

  • \(C_{\text{catch}}\) represents successful error identification
  • \(C_{\text{explain}}\) represents explanation quality
  • \(C_{\text{apply}}\) represents successful transfer to a new problem
  • \(P_{\text{false}}\) penalizes incorrect corrections

Most importantly: high confidence requires successful application to a new problem. A student who can recognize a mistake but cannot transfer the concept is still developing.

What Makes CounterTutor an Agent?

A normal chatbot primarily responds to the latest message. CounterTutor maintains state and makes decisions based on accumulated evidence.

Student catches misconception
        ↓
Student explains it
        ↓
CounterTutor remembers
        ↓
Student catches it again
        ↓
CounterTutor changes strategy
        ↓
    TRANSFER
        ↓
New unseen problem

The important decision is not generated randomly by the LLM. The deterministic agent sees the accumulated state and decides what should happen next. For example:

"You've caught this misconception twice. I'm not going to ask you to recognize it again. Let's see if you can apply it yourself."

That behavior is driven by the student's stored evidence.

Technical Architecture

             ┌─────────────────────┐
             │       Frontend       │
             │   Student Chat UI    │
             └──────────┬───────────┘
                         ↓
             ┌─────────────────────┐
             │     Backend API      │
             │    Session State     │
             └──────────┬───────────┘
                         ↓
             ┌─────────────────────┐
             │    Agent Engine       │
             │  State + Decisions    │
             │       No LLM          │
             └───────┬───────┬───────┘
                      │       │
              ┌───────┘       └────────┐
              ↓                        ↓
       ┌──────────────┐        ┌───────────────┐
       │ Curated SAT   │        │      LLM       │
       │ Content Bank  │        │   Evidence      │
       │               │        │   Extraction    │
       └──────────────┘        └───────────────┘

Separation of responsibilities

  • Curated content — Owns the educational truth.
  • LLM — Understands natural-language student explanations.
  • Agent Engine — Owns state, mastery, and decisions.
  • Backend — Orchestrates the interaction.
  • Frontend — Provides the student experience.

This separation makes the system more predictable and easier to test.

Challenges We Ran Into

Making It Genuinely Agentic

The biggest challenge was avoiding the trap of building a chatbot with an impressive prompt. An LLM saying "I will adapt to your learning!" doesn't make an agent. We moved the important decisions into deterministic state-machine logic. The system explicitly tracks state and decides what happens next.

Preventing Students From Gaming the AI

If the AI were wrong every time, students could simply assume the AI's answer is always wrong. So CounterTutor includes both incorrect and correct AI attempts. The student must actually evaluate the reasoning.

Catching Isn't the Same as Understanding

A student can notice that something looks suspicious without understanding why. That led us to separate Catch → Explain → Apply. The transfer step became essential because it gives us stronger evidence that the student understood the concept.

Keeping the System Reliable

We intentionally kept educational truth outside the LLM. SAT misconceptions and questions are curated. Transfer questions have deterministic answer keys. The LLM only interprets natural-language reasoning. If the evaluator produces invalid output, the system falls back to a safe PROBE action instead of silently advancing the student's mastery.

Accomplishments We're Proud Of

We flipped the traditional AI tutoring relationship. Instead of AI → Student, CounterTutor creates Student → AI. The student becomes the person doing the explaining.

The agent actually remembers what happened. The system does not simply respond to the latest message — it accumulates evidence and changes its behavior:

"You've caught this misconception twice. I'm not going to ask you to recognize it again. Let's see if you can apply it yourself."

That decision comes from stored session state. It isn't a random LLM response.

We made the core agent independently testable. The state machine can be tested with zero LLM calls. That gives us predictable behavior, easier debugging, and a clear separation between AI-generated language and actual agent logic.

We kept accessibility in the architecture. CounterTutor uses short interactions, curated content, and only one short LLM evaluation call per student response. There are no unnecessary video feeds, heavy simulations, or complex infrastructure requirements. The core learning experience remains a lightweight conversation.

What We Learned

The best AI education experience may not be the AI that teaches the most. It may be the AI that makes the student think the most.

We also learned that an "AI agent" should mean more than an LLM with a system prompt. A useful agent needs:

STATE
  ↓
EVIDENCE
  ↓
DECISION
  ↓
ACTION
  ↓
NEW STATE

That distinction shaped the entire architecture of CounterTutor.

We also learned that measuring learning is harder than measuring correctness. A correct answer tells us what happened. A student's explanation gives us clues about why it happened. Transfer gives us evidence that the understanding survived a change in context.

Accessibility

CounterTutor was designed with students using limited devices and unreliable connectivity in mind. The MVP intentionally avoids heavy media and unnecessary infrastructure. The interaction uses:

  • lightweight text
  • short LLM requests
  • curated question content
  • deterministic decision logic
  • minimal frontend payloads
  • graceful fallback when the LLM evaluator fails

The architecture does not depend on video, audio, complex simulations, or large client-side models.

The goal is simple: the intelligence should be useful even when the hardware isn't powerful.

What's Next for CounterTutor

The current MVP focuses on SAT Math and a curated set of common misconceptions. The next stage would expand the same learning architecture rather than simply adding more features.

Future directions:

  • Expand the misconception library across the SAT Math curriculum
  • Add SAT Reading and Writing using the same Catch → Explain → Apply model
  • Build longer-term student memory across study sessions
  • Improve misconception detection using accumulated interaction evidence
  • Add optional voice interaction for students who learn better by speaking
  • Explore teacher-facing summaries based on demonstrated misconceptions rather than simple quiz scores

The core principle stays the same:

Don't just tell students whether they are right. Make them understand why.

The CounterTutor Loop

              ┌─────────┐
              │  CATCH  │
              └────┬────┘
                   ↓
              ┌─────────┐
              │ EXPLAIN │
              └────┬────┘
                   ↓
              ┌─────────┐
              │  APPLY  │
              └────┬────┘
                   ↓
              ┌─────────┐
              │ REMEMBER│
              └────┬────┘
                   ↓
              ┌─────────┐
              │  ADAPT  │
              └────┬────┘
                   │
                   └──────────────→ NEXT LEARNING CHALLENGE

CounterTutor turns the student from an answer generator into an active reasoner.

Built With

  • accessibility
  • adaptivelearning
  • agenticai
  • ai
  • aiagents
  • edtech
  • education
  • framermotion
  • genai
  • glassmorphism
  • humanai
  • learning
  • natural-language-processing
  • next.js
  • openai
  • personalizedlearning
  • react
  • sat
  • statemachine
  • student
  • tailwindcss
  • tutoring
  • typescript
  • webapp
Share this project:

Updates