2. Project Description --- CritiqAI
Problem Statement
The Problem: AI Is Making It Easier to Avoid Thinking
Generative AI has made it extremely easy for students to produce polished- looking essays and answers. However, a polished output does not necessarily mean the student understands the argument be A student can submit an essay that looks sophisticated while being unable to defend its claims when a teacher asks a simple follow-up question. Weak evidence, unsupported assumptions, missing counterarguments, logical gaps, and overly narrow conclusions can remain hidden because conventional AI tools are optimized to produce or improve answers, rather than make students examine their own reasoning.
This creates a deeper educational problem:
- Students can outsource parts of the thinking process to AI.
- AI-generated improvements can hide the student's actual weaknesses.
- A polished essay can create false confidence.
- Students may become better at producing answers without becoming better at defending or evaluating them.
- Teachers have less visibility into whether a student genuinely understands and can reason about their own argument.
CritiqAI treats this as a design problem rather than a student failure. Instead of building another AI system that writes better answers for students, CritiqAI uses AI to create the cognitive friction that students need in order to practice reasoning.
Core Principle
"Using AI to teach students not to depend on AI."
CritiqAI therefore follows a strict educational rule:
The AI challenges the student's thinking; it does not think on the student's behalf.
The system does not rewrite the essay, provide corrected arguments, reveal the ideal answer, or simply praise the student's work. Instead, it identifies weaknesses and asks increasingly difficult questions that force the student to defend, clarify, and reconsider their own reasoning.
Solution Overview
What Is CritiqAI?
CritiqAI is a multi-agent AI debate coach built with Google ADK that turns a student's essay into a structured, three-round Socratic debate.
The system analyzes the structure of an essay, identifies its most important reasoning weaknesses, selects an adversarial AI persona specifically suited to those weaknesses, and challenges the student through three escalating rounds.
After the debate, the student's responses are evaluated across four critical-thinking dimensions:
- Logical Coherence
- Evidence Quality
- Counterargument Handling
- Scope Awareness
The system then converts the results into structured analytics and prepares a teacher report. The report is placed into a Gmail draft, but the teacher must review and manually send it.
This creates a complete workflow in which AI is used as a reasoning challenger and learning tool, while important decisions and communication remain under human control.
How the Solution Works
At a high level, CritiqAI follows this pipeline:
┌─────────────────────────┐
│ Student Essay │
│ Google Drive / Text │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ Input Processing │
│ │
│ • Sanitize essay │
│ • 2000-word limit │
│ • Detect language │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ Summarizer Agent │
│ │
│ Extracts: │
│ • Main claim │
│ • Supporting points │
│ • Evidence │
│ • Conclusion │
└────────────┬────────────┘
│
▼
┌─────────────────────────┐
│ Persona Selector Agent │
│ │
│ Identifies weaknesses │
│ and selects challenger │
│ persona(s) │
└────────────┬────────────┘
│
▼
┌────────────────────────────────────────┐
│ Debate Agent │
│ │
│ Round 1 → Initial challenge │
│ Round 2 → Pressure weakest response │
│ Round 3 → Push deeper / expose limits │
└───────────────────┬────────────────────┘
│
▼
┌─────────────────────────┐
│ Challenge Validator │
│ │
│ • Answer-leak check │
│ • Single-question rule │
│ • 150-word ceiling │
│ • 18 leak patterns │
└────────────┬────────────┘
│
┌────────────┴────────────┐
│ │
FAIL PASS
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Regenerate once │ │ Challenge shown │
│ │ │ to student │
└────────┬─────────┘ └────────┬─────────┘
│ │
└──────────────┐ ▼
│ ┌──────────────────┐
│ │ Student Response │
│ └────────┬─────────┘
│ │
└─────────────┤
▼
┌────────────────────────┐
│ Repeat Debate Rounds │
│ 1 → 2 → 3 │
└────────────┬───────────┘
│
▼
┌────────────────────────┐
│ argument-scorer MCP │
│ │
│ Paul-Elder dimensions: │
│ • Logic │
│ • Evidence │
│ • Counterarguments │
│ • Scope │
└────────────┬───────────┘
│
▼
┌────────────────────────┐
│ Analytics Agent │
│ │
│ Scores + strengths + │
│ weaknesses + suggestions│
└────────────┬───────────┘
│
▼
┌────────────────────────┐
│ Report Agent │
│ │
│ Creates teacher report │
└────────────┬───────────┘
│
▼
┌────────────────────────┐
│ Gmail MCP │
│ Create Draft Only │
└────────────┬───────────┘
│
▼
┌────────────────────────┐
│ Teacher Reviews │
│ and Manually Sends │
└────────────────────────┘
Key Features
1. Six Specialized AI Agents
CritiqAI uses separate agents instead of putting the entire workflow into one large prompt.
Orchestrator
Acts as the root of the ADK pipeline and manages routing and session state.
Summarizer Agent
Compresses the student's essay into a structured representation containing the main claim, supporting points, evidence, and conclusion.
This reduces the amount of information that downstream agents need to process and helps control token usage.
Persona Selector Agent
Analyzes the summarized argument and determines which adversarial perspective is most useful for challenging the student's reasoning.
Debate Agent
Runs the three-round Socratic debate and generates one focused challenge at a time.
Challenge Validator Agent
Acts as a deterministic quality and safety gate. It checks every generated challenge before it reaches the student.
Analytics + Report Agent
Aggregates the scoring results, identifies strengths and weaknesses, and prepares the teacher-facing report.
2. Four Adversarial Debate Personas
The Persona Selector maps weaknesses in the student's reasoning to specialized challengers.
Persona Primary Focus What It Challenges
The Skeptic Evidence Unsupported claims, source credibility, methodology, sample size, cherry-picking
The Devil's Counterarguments Opposing viewpoints, Advocate counterexamples, alternative explanations
The Nitpicker Logical consistency Contradictions, ambiguous terms, precision, premises vs. conclusions
The Expander Scope Missing populations, contexts, edge cases, hidden assumptions
For example:
Weak evidence
↓
The Skeptic
Missing opposing views
↓
The Devil's Advocate
Internal contradiction
↓
The Nitpicker
Argument too narrow
↓
The Expander
The persona-selection logic is represented as a dedicated Agent Skill so that the educational decision process can be maintained independently from the core agent code.
3. Three-Round Escalating Debate
CritiqAI does not simply ask three unrelated questions.
The debate is designed to become progressively more demanding:
Round 1
↓
Clarify the argument
Request basic evidence
Understand the student's position
↓
Round 2
↓
Challenge the weakest response
Introduce counterarguments
Demand greater precision
↓
Round 3
↓
Push on breadth and depth
Expose remaining assumptions
Force the student to confront limitations
The student's responses become part of the debate history, allowing later challenges to respond to what the student actually said.
4. Deterministic Challenge Validation
One of the core safety mechanisms is the Challenge Validator Agent.
The Debate Agent is allowed to generate challenges, but those challenges are not trusted automatically.
Every challenge is checked using pure Python before reaching the student.
The validator checks:
- 18 answer-leak patterns
- Whether the challenge contains too many questions
- Whether the challenge exceeds the 150-word ceiling
The most important rule is preventing the AI from accidentally telling the student what the answer should be.
The architecture therefore separates generation from enforcement:
LLM
│
│ generates challenge
▼
Deterministic Python Validator
│
├── PASS → Student
│
└── FAIL → Regenerate once
This approach requires zero LLM tokens for validation and makes the safety logic auditable.
5. Critical-Thinking Scoring
After the debate, the student's responses are scored using four Paul-Elder dimensions.
Logical Coherence
Measures whether the student's claims logically follow from their premises.
Evidence Quality
Measures the quality and concreteness of the evidence supporting the argument.
Counterargument Handling
Measures whether the student recognizes and meaningfully engages with opposing views.
Scope Awareness
Measures whether the student recognizes limitations, assumptions, edge cases, and boundaries of the argument.
Each dimension receives a score from 0 to 5.
The resulting structure can be represented as:
Logical Coherence 0–5
Evidence Quality 0–5
Counterargument Handling 0–5
Scope Awareness 0–5
----
Maximum 20
6. Hybrid Scoring for Multilingual Responses
The custom argument-scorer MCP uses a hybrid strategy.
English
English responses are scored using deterministic keyword-based rules.
English response
↓
Python scoring rules
↓
0 LLM tokens
This makes the scoring inexpensive and auditable.
Vietnamese / Japanese / Chinese
For supported non-English responses, the scorer uses one compact Gemini call to score all four dimensions together.
The approach is designed to avoid making a separate LLM request for every scoring dimension.
The non-English scoring path also uses deterministic temperature settings and LRU caching.
7. Google MCP Integrations
CritiqAI uses MCP to connect the agent system to external services with scoped responsibilities.
Google Drive MCP
Used to read student essay submissions.
The documented access is read-only and restricted to the submissions area.
Google Sheets MCP
Used to append debate history and scoring information for auditability.
Gmail MCP
Used only to create a teacher report draft.
The system does not autonomously send the email.
Custom argument-scorer MCP
A custom FastMCP service responsible for critical-thinking evaluation.
It can run as a Cloud Run service and also has local development fallbacks.
8. Human-in-the-Loop Teacher Approval
The final teacher communication is intentionally not fully automated.
The workflow is:
Debate
↓
Analytics
↓
Teacher Report
↓
Gmail Draft
↓
Teacher Opens Draft
↓
Teacher Reviews Content
↓
Teacher Manually Sends
This ensures that a human educator remains responsible for reviewing the report before it is communicated.
The system therefore automates the preparation of information while keeping the final external communication under human control.
9. Security and Responsible AI
CritiqAI incorporates security controls directly into its architecture.
Input Validation
Student input is sanitized and limited to 2,000 words before LLM processing.
Secrets Management
Credentials are supplied through environment variables rather than hardcoded into the application.
Least-Privilege Access
Different integrations receive only the access required for their specific purpose.
Drive → Read-only
Sheets → Controlled spreadsheet access
Gmail → Compose only
Behavioral Monitoring
Every AI-generated debate challenge passes through the deterministic validator.
Human Oversight
Teacher communication requires manual approval.
Together, these controls are intended to prevent both technical misuse and undesirable AI behavior.
Target Users
Primary Users
Students
Students use CritiqAI to practice:
- defending their own arguments
- evaluating evidence
- handling opposing viewpoints
- recognizing logical weaknesses
- understanding the limits of their conclusions
The student is not given a polished replacement answer. The system instead creates an environment where the student has to reason for themselves.
Teachers
Teachers receive structured information about how a student handled challenges, including:
- critical-thinking scores
- strengths
- weaknesses
- improvement suggestions
- debate history
- a prepared report for review
This gives teachers another way to see how well students can defend the reasoning behind their written work.
Example User Journey
Consider a student who submits:
"Social media should be banned for users under 18 because studies show it causes depression and companies only care about profit."
The system may identify the primary weakness as insufficient evidence.
The Persona Selector therefore selects:
The Skeptic
Round 1
The student is challenged to identify and explain the evidence behind the claim.
Round 2
The system introduces evidence or a perspective that requires the student to consider potential positive effects or exceptions.
Round 3
The challenge shifts toward the student's policy conclusion and asks whether an immediate ban is the only possible response.
The student is therefore pushed through:
Evidence
↓
Counterevidence
↓
Policy scope
After the debate, the student's performance is scored across the four critical-thinking dimensions.
The result can reveal not only a final score but where the reasoning improved and where weaknesses remain.
Technology Used
CritiqAI is built around the following technologies and architectural components:
Technology / Component Role
Google ADK Multi-agent orchestration and agent execution
Gemini 2.5 Flash Lite Summarization, persona selection, and debate generation
FastAPI Web application and teacher/student interface
MCP Controlled connections between agents and external tools/services
FastMCP Custom argument-scorer server
Google Drive Essay submission/retrieval
Google Sheets Debate logging and persistence
Gmail Teacher report draft creation
Google Cloud Run Deployment of the custom argument-scorer service
Python Core application, validation, analytics, scoring, and integrations
Paul-Elder Framework Critical-thinking scoring dimensions
Agent Skills Versioned pedagogical rules and debate behavior
Why the Multi-Agent Architecture Matters
The system deliberately separates responsibilities instead of asking one AI agent to perform everything.
A monolithic design would have to simultaneously:
Read essay
+
Understand argument
+
Identify weaknesses
+
Choose persona
+
Maintain debate state
+
Generate challenges
+
Avoid giving answers
+
Score responses
+
Create reports
This creates conflicting responsibilities and makes individual behaviors harder to test.
CritiqAI instead separates the responsibilities:
Summarize
↓
Select Persona
↓
Debate
↓
Validate
↓
Score
↓
Analyze
↓
Report
Each component has a narrower responsibility, making the system easier to evaluate, audit, and modify.
Overall System Architecture
CRITIQAI
│
▼
┌────────────────────┐
│ Student / Teacher │
│ FastAPI UI │
└─────────┬──────────┘
│
▼
┌────────────────────┐
│ Session Manager │
│ Shared State │
└─────────┬──────────┘
│
▼
┌─────────────────────────────┐
│ Google ADK Pipeline │
│ │
│ ┌───────────────────────┐ │
│ │ Summarizer │ │
│ └───────────┬───────────┘ │
│ ▼ │
│ ┌───────────────────────┐ │
│ │ Persona Selector │ │
│ └───────────┬───────────┘ │
│ ▼ │
│ ┌───────────────────────┐ │
│ │ Debate Agent │◄──────┐
│ │ 3 Rounds │ │
│ └───────────┬───────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────┐ │
│ │ Challenge Validator │───────┘
│ │ Deterministic Python │
│ └───────────┬───────────┘
│ │
│ ▼
│ Student Response
│ │
└──────────────┼─────────────────────
│
▼
┌─────────────────────┐
│ argument-scorer MCP │
│ │
│ 4 dimensions × 0–5 │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Analytics Agent │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Report Agent │
└──────────┬──────────┘
│
▼
┌─────────────────┐
│ Gmail MCP │
│ create_draft │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Teacher Review │
│ + Manual Send │
└─────────────────┘
External / Supporting MCP Services
──────────────────────────────────
Google Drive MCP
│
└──► Essay Retrieval
Google Sheets MCP
│
└──► Debate Log / Scores
Gmail MCP
│
└──► Teacher Draft
argument-scorer MCP
│
└──► Critical-Thinking Scores
The Core Value Proposition
CritiqAI does not attempt to make students better at asking AI for answers.
It attempts to make students better at:
- defending what they believe
- examining evidence
- responding to opposing arguments
- recognizing contradictions
- identifying assumptions
- understanding the scope of their claims
- recognizing where their own reasoning breaks down
The central transformation is therefore:
Traditional AI Education Tool
Student → AI → Answer
↓
Less cognitive effort
CritiqAI
Student → AI Challenge → Student Reasoning
↑ ↓
└──── Feedback ───┘
More cognitive engagement
CritiqAI's goal is not to eliminate AI from education. It is to change what the AI is doing.
Instead of becoming the student's substitute thinker, the AI becomes the student's adversarial thinking partner.
Built With
- agent-skills
- cloud-run
- docker
- fastapi
- fastmcp
- gemini
- generative-ai
- gmail
- google-cloud-run
- google-drive
- googleadk
- googlesheets
- mcp
- multi-agentic-ai
- paul-elder-criticaal-thinking-framework
- python
Log in or sign up for Devpost to join the conversation.