Misconception Studio

Misconception Studio is a teacher-controlled active evidence loop for low-stakes formative assessment. Instead of jumping from a wrong answer to a diagnosis, it asks a more useful question: what evidence would change our mind?

The problem

Most classroom response tools can show that many learners chose the same wrong answer, but the same answer can come from different reasoning. An AI system that immediately labels a misconception can turn an ambiguous signal into an overconfident instructional decision.

Misconception Studio keeps the teacher in control and treats every interpretation as a contestable hypothesis.

What the current build does

The implemented backend vertical slice starts with registered, adult-authored synthetic fractions responses and runs a bounded evidence loop:

1. A capability-scoped teacher demo session loads a registered synthetic fixture and screens the selected response through a strict input policy.
2. Three stateless, structured-output roles run in sequence through the OpenAI Responses API: Analyst → Challenger → Probe Designer.
3. The Analyst grounds a candidate claim in literal response spans. The Challenger names a plausible alternative. The Probe Designer creates one targeted follow-up question intended to distinguish between them.
4. Strict schema and manifest validation reject fabricated spans, unknown IDs, invalid claim links, and out-of-policy outputs.
5. Deterministic domain verification checks the generated probe before delivery. A probe can be routed for teacher approval, edited and reverified, automatically delivered only when every certification and teacher opt-in gate is satisfied, or replaced by a reviewed fallback.
6. A registered synthetic follow-up response enters the existing deterministic evidence gate, which is the only component allowed to change evidence state.
7. The system records a replay-safe audit trail so the teacher can inspect the hypothesis, counterevidence, probe provenance, delivery decision, and resulting state transition.

Teacher authority

The model never writes evidence state and never silently decides the instructional action. Teachers can approve a verified probe, edit it, or reset the session. Any edit invalidates the previous verification and must pass deterministic checks again. Missing certification, validation failure, timeout, refusal, or provider error routes to teacher review or a reviewed fallback without changing the evidence state.

Reliability and privacy boundaries

The current API is fixture-ID-only: it exposes no arbitrary learner-text field, student accounts, class-list import, or classroom upload. Sessions use HttpOnly capability cookies and trusted-origin checks. Raw response text, literal excerpts, natural-key mappings, and capability material are separated from the privacy-safe metadata ledger, expire within 24 hours, and are hard-deleted on reset. Model calls are stateless, use structured outputs, disable storage, perform no hidden retries, and operate under fixed request, token, and time ceilings.

Fresh local checks on the latest implementation passed 261 unit tests, 6 architecture tests, 13 PostgreSQL integration tests, and strict type checking across 44 source files.

How GPT-5.6 is used

The live adapter uses GPT-5.6 (gpt-5.6-terra) through the OpenAI Responses API for the three bounded Toulmin roles. Each role receives only the allowed manifest vocabulary and returns a typed object. Deterministic code—not the model—validates spans, IDs, domain constraints, delivery eligibility, and evidence-state transitions.

Built with Codex and Delapan

Codex was used to turn the product research and safety constraints into executable architecture, implement the backend in small reviewed increments, generate tests, and harden replay, privacy, and teacher-control boundaries. Delapan supported the evidence-backed competition and product research that shaped the concept and claim boundaries.

Current scope and limitations

This is a synthetic, adult-authored hackathon demonstration. It establishes an inspectable software and evidence-control architecture; it does not establish diagnostic accuracy, educational efficacy, fairness, classroom readiness, or production compliance.

The backend evidence loop is implemented. Remaining release work includes the teacher console, an independently reviewed runtime content pack, final generated API-contract synchronization and cleanup, deployment, evaluator artifacts, demo video, and the final Codex feedback session.

Built With

• alembic
• codex
• fastapi
• gpt-5.6
• mypy
• openai-responses-api
• openai-structured-outputs
• postgresql-17
• pydantic
• pytest
• python-3.13
• sqlalchemy