Crucible

• 

  Codex live laboratory

• 

  Machine-verified execution evidence

• 

  Evidence-based model update

• 

  Transfer challenge

• 

  Architecture

Inspiration

An advanced learner can explain a concept fluently and still carry one hidden assumption that fails under pressure. Our original idea was weakness-focused learning: identify misconceptions, deliberately practice the fragile parts, and let learners choose the sources they want to master.

We realized that a frontier model should do much more than generate harder quizzes. The more interesting capability is experimental: form multiple explanations for a learner's answer, find an observation that separates them, build the instrument needed to obtain that observation, and revise the diagnosis when evidence arrives.

That became Crucible. A crucible applies pressure that reveals what a material is actually made of. This product applies controlled pressure to an explanation until shallow familiarity separates from transferable understanding.

What it does

Crucible is an adversarial learning laboratory for advanced technical concepts. Its flagship journey has eight stages:

1. THE FORGE — choose a grounded Transformer source or paste bounded technical material and define a mastery objective.
2. CONCEPT ATLAS — reconstruct claims, prerequisites, assumptions, and fragile boundaries.
3. DIAGNOSTIC — commit a free-form prediction before observing the result.
4. HYPOTHESES — maintain competing mental models that could explain the learner's answer.
5. LIVE LAB — turn the most informative distinction into one bounded Python artifact, declared invariants, and a structured-output contract.
6. EVIDENCE — execute the laboratory in a disposable Docker environment and compare the prediction with exact output.
7. MODEL UPDATE — revise the hypothesis ledger only where evidence earns an update.
8. TRANSFER — test the revised distinction in an unfamiliar context.

The flagship examines a subtle Transformer claim: whether equal total width makes one conventional attention head functionally equivalent to multiple independently routed heads. Instead of grading vocabulary, Crucible constructs and executes the counterexample that exposes exactly where the learner's rule fails—and carefully states what the experiment does not prove.

Why this is not another AI quiz app

A conventional tutor asks a question, compares an answer with a rubric, explains the expected answer, and adjusts difficulty. Crucible treats the response as evidence about a hidden causal model.

Its frontier loop is:

reconstruct → commit → hypothesize → discriminate → build → execute → update → transfer

The decisive output is executable evidence, not a second model opinion. The interface also distinguishes source-bounded material, GPT-5.6 inference, generated artifacts, execution observations, and separately checked flagship results.

How we built it

The key separation: GPT-5.6 proposes and revises the semantic model; Codex materializes a schema-constrained falsifier; the local runner limits authority; and isolated execution returns exact evidence that can overturn the diagnosis.

The product has two complementary paths.

The hosted showcase is a frictionless interactive presentation of the checked Transformer journey, explicitly labeled VERIFIED REPLAY. It requires no visitor credentials and never pretends to inherit a user's ChatGPT session or perform a fresh model call.

The full local experience uses a host-side runner bound to 127.0.0.1. It invokes the user's authenticated Codex CLI through schema-constrained non-interactive execution. Generated laboratory code is materialized at a fixed path and executed in a disposable Docker container with no Codex credentials, no Docker socket, bounded resources, and no network by default. Exact structured evidence returns to a distinct GPT-5.6 update stage.

The web experience uses Next.js and React. The flagship numerical lab compares a shared attention distribution with independently controlled distributions under precisely stated constraints. Automated tests cover the runner API, flagship experiment, rendered interface, types, lint, and production build.

How we used Codex

Codex was the implementation collaborator throughout Build Week. We used it to translate the thesis into an executable architecture, implement the eight-stage interface, build the local runner and structured contracts, create the Transformer laboratory and replay path, engineer the Docker boundary, write and execute tests, diagnose failures, document the security and evidence model, and produce the under-three-minute working-product demo.

The human participant originated the weakness-focused learning problem, rejected shallow chatbot and quiz variants, selected the audience and flagship domain, challenged overbroad claims, reviewed the output, and retains responsibility for the submission.

How we used GPT-5.6

In live local mode, GPT-5.6 performs semantic work that a static question bank cannot:

• reconstruct an unfamiliar concept from supplied material;
• interpret free-form learner reasoning;
• generate and rank competing misconception hypotheses;
• identify a high-information discriminating experiment;
• create a structured experiment and executable artifact;
• inspect external execution evidence;
• revise the diagnosis; and
• create a changed-context transfer challenge.

The runner uses three schema-constrained calls and defaults to configurable xhigh reasoning effort. Structured schemas, static policy screening, bounded outputs, and exact Docker execution form the harness around the model. Docker proves that an artifact ran and reports its output; it does not magically certify the academic interpretation for every arbitrary source.

Challenges we ran into

Showing frontier intelligence without exposing hidden reasoning

We wanted the model's work to be legible without presenting private chain-of-thought or a decorative token stream. We expose explicit artifacts instead: concept claims, competing hypotheses, supporting and contradicting evidence, a falsifier rationale, code, invariants, and a diagnosis delta.

Making the demo authentic and reliable

A hosted page cannot inherit a visitor's private Codex session, while fresh inference latency is risky in a short judging video. We separated an honestly labeled replay from a bring-your-own-Codex local runner. The demo includes an authentic full-HD browser recording of Crucible's verified execution replay and labels it truthfully.

Preventing the model from grading itself

An eloquent explanation of generated code is not verification. We created a narrow numerical experiment with declared invariants and external execution observation, then scoped the resulting claim instead of overstating it as a universal theorem.

Treating generated code as untrusted

Generated labs run separately from Codex credentials and host authority. The Build Week sandbox is still a prototype, so we document residual risks rather than claiming perfect isolation.

Accomplishments we are proud of

• Crucible investigates a hidden model of understanding instead of matching prose to an answer key.
• The flagship misconception is sophisticated enough to sound plausible to a technically informed learner.
• GPT-5.6 owns an evidence-seeking loop rather than a single response.
• Codex creates a real, inspectable, executable falsifier.
• The interface makes epistemic status visible.
• The product is frictionless to inspect and authentic to run locally.
• The demo remains narrow, falsifiable, and honest.

What we learned

The strongest use of a frontier model is not always answering the hardest question directly. It is owning an evidence-seeking loop: understand an unfamiliar environment, state uncertainty, create an artifact, obtain external feedback, and change course when the evidence disagrees.

Advanced learners do not only need more difficult content. They need experiments targeted at the distinction their current reasoning leaves unresolved. Personalization becomes meaningful when it changes what evidence the system seeks, not merely the reading level of an explanation.

We also learned that harness engineering is part of the intelligence: schemas, provenance, sandboxing, deterministic checks, observable state, and graceful failure are what make an agent's capability credible.

What's next

We want to validate the diagnostic method with advanced learners and educators; calibrate hypothesis scores; add paper and repository ingestion with stronger provenance; develop domain-specific verifiers for distributed systems, quantum circuits, and compiler behavior; and study whether transfer-verified challenges predict durable understanding.

Crucible will not claim universal domain support until each domain has a credible evidence mechanism.

Built With

• codex
• openai
• python
• react