Judgment Mirror
AI outputs are observable. Judgment shifts usually are not.
AI systems leave visible traces: prompts, responses, recommendations, and citations. The human decision that encounters that output usually leaves almost none.
The AI output can be inspected. What usually remains invisible is whether a person became more confident, changed direction, rejected the AI’s advice, adopted it, or moved beyond it without gaining additional evidence.
Judgment Mirror makes that interaction visible.
Inspiration
AI evaluation typically focuses on the model: Was its answer accurate, safe, persuasive, or well-reasoned?
That matters, but it leaves out half of the interaction.
Judgment Mirror began with a different question:
What changed in a person’s decision process after the AI answered?
Judgment Mirror is an experimental observatory for human judgment during AI interaction. It is not an intelligence test, psychological assessment, truth detector, or decision-grading system. It does not determine whether the user or the AI is correct. It visualizes the decision process that occurred during one interaction.
What Judgment Mirror does
Judgment Mirror guides the user through one realistic operational decision scenario.
Before seeing the AI response, the user records:
- An independent decision
- Confidence from 0–100
- The primary reason for the decision
- Information that could change the decision
GPT-5.6 then produces a credible recommendation containing a clear decision, supporting reasons, verification questions, and a material caveat.
After reading the recommendation, the user records:
- A post-AI decision
- Post-AI confidence
- The AI statement that was most influential
- Whether any claims were verified
Judgment Mirror then displays the interaction as four separate states:
- Pre-AI decision
- AI recommendation
- Post-AI decision
- Final reflection decision
The report shows whether the user maintained the original decision, matched the AI recommendation, rejected it, or moved beyond it.
It also displays Confidence Delta, Evidence Delta when calculable, and a clearly labeled prototype Post-AI Judgment–Evidence Gap.
Selecting an AI statement as influential does not convert it into a user-authored claim or verified fact. Claims remain attached to the exact stage where they appeared.
When there are no user-authored material claims in the post-AI stage, evidence coverage is displayed as Not calculable. Judgment Mirror does not misrepresent a missing denominator as zero evidence.
The interaction ends with a neutral reflection question and one final opportunity to retain, revise, or reconsider the decision.
How I built it
I built Judgment Mirror as a standalone Next.js and TypeScript application with Codex and the OpenAI Responses API using GPT-5.6.
I defined the product protocol, scientific boundaries, acceptance standards, failure cases, and required integrity rules. Codex served as my implementation partner, translating those requirements into a working interface, structured schemas, deterministic calculations, reliability controls, and regression tests.
GPT-5.6 performs two bounded tasks:
- Generating the AI recommendation
- Analyzing the visible differences between the recorded judgment stages
Both calls require structured JSON outputs validated through strict schemas.
The model does not control the locked user snapshots, decision comparisons, evidence-coverage arithmetic, metric calculations, or stage attribution. Those are enforced through deterministic application logic.
The prototype stores the complete session only in browser state. It contains no authentication, database, user profile, payment system, uploads, or unrelated features.
I used Codex throughout the build to implement and refine:
- The stage-controlled interface
- Structured-output schemas
- Deterministic integrity processing
- Scenario-specific decision comparison
- Claim-provenance rules
- Evidence-coverage calculations
- Bounded timeout and retry behavior
- Regression and integration fixtures
- Scientific-restraint language across the interface
Challenges I faced
The hardest problem was not generating a persuasive AI recommendation. It was preventing that recommendation from contaminating the observation of the user’s decision.
I had to ensure that:
- The AI recommendation never became the user’s selected decision
- An influential AI statement never became a user-authored claim
- An influential statement was not treated as independently verified
- Unsupported claims remained attached to their original stage
- Final reflection never rewrote the post-AI snapshot
- Missing evidence was not misrepresented as zero evidence
- Reflection language remained neutral rather than prosecutorial
- Variable model language could not create inaccurate decision comparisons
- All four decision states remained separate and unchanged
A second challenge was real-world latency.
Live testing exposed an analysis request that could stall while the interface remained indefinitely on “Building your mirror…”
I used Codex to diagnose the failure, add immediate server-side timing logs, disable invisible SDK retries, implement one bounded application-controlled retry, add a client timeout, preserve every locked answer, prevent duplicate requests, and provide a visible recovery path.
A third challenge was maintaining scientific restraint.
Judgment Mirror needed to make the interaction memorable without presenting an experimental product metric as established science. The interface therefore identifies the Post-AI Judgment–Evidence Gap as a prototype metric and clearly states what the system does not claim.
What I learned
I learned that human–AI observability requires more than asking another model to analyze the interaction.
The system must strictly separate:
- AI-generated statements
- User-authored claims
- Statements selected as influential
- Scenario-supported facts
- User-reported verification
- Unsupported claims
- Inferences
- Deterministic calculations
I also learned that scientific restraint must exist in the product logic—not only in a disclaimer.
A missing denominator must remain Not calculable. A prototype metric must remain labeled as a prototype metric. A decision shift must be described without assuming that AI strengthened or weakened judgment.
Codex accelerated my implementation, but its greatest value was iterative verification. Each live failure or conceptual inconsistency could be translated into a deterministic test, integrity rule, or bounded recovery path.
Validation
The final prototype passed:
- 80 automated tests
- TypeScript validation
- ESLint with zero warnings
- Production build validation
- The original failure-pattern integration fixture
- Three complete live Scenario 01 journeys
All three live journeys rendered one complete Judgment Mirror without manual retry. Every run preserved all four decision states independently, retained AI-generated provenance, used neutral reflection language, and correctly handled the no-claim evidence state.
What comes next
The next stage is not to add more participant-facing metrics and call them science.
Judgment Mirror would require controlled pilot studies, versioned scenarios, preregistered hypotheses, reliability testing, construct validation, independent replication, and clearly bounded scientific claims before it could become a validated research instrument.
This prototype establishes the interaction protocol and observability foundation required to begin that work.
Limitations
Judgment Mirror is experimental and not scientifically calibrated.
It does not:
- Evaluate intelligence
- Evaluate reasoning ability
- Diagnose cognition or mental health
- Determine whether a decision is correct
- Determine whether the user or AI is trustworthy
- Independently confirm external evidence
- Predict future behavior
- Replace professional decision systems
Judgment Mirror observes only the information recorded during one AI-assisted decision process.
Its purpose is visibility—not judgment of the person.
Log in or sign up for Devpost to join the conversation.