ChalkTrace - proof-carrying adaptive learning system

Inspiration

Generated tutoring content has an asymmetric failure mode: a polished activity can still be wrong, weakly sourced, answer-leaking, or badly matched to the learner, and it fails quietly. One wrong practice item in a 1,200-seat course is 1,200 wrong learning events. I wanted to answer a stricter question than "can a model write a lesson?" — what must an AI learning system prove before an activity is allowed to shape a learner's future?

That produced two product primitives: proof-carrying learning objects, and challengeable adaptation.

What it does

ChalkTrace is a durable, multi-organization course studio plus a signed sample learning workspace.

An educator registers, creates a course, and imports their own material — text-bearing PDFs, IMS Common Cartridge / Canvas .imscc, Moodle .mbz, ZIP, QTI/XML, HTML, Markdown, or plain text. The upload is parsed as inert data behind archive-traversal, symlink, zip-bomb, and XML defenses, then split into content-addressed source regions. A lesson candidate — drafted deterministically or proposed by GPT-5.6 — carries a public activity and a separate, server-only grading contract. Learners and browsers never receive canonical answers or private grading rationales.

Publication runs an eight-check policy against the exact candidate bytes: schema, source support, executable correctness, adversarial solvability, hint leakage, accessibility, runtime safety, and an independent educator review where an authorized educator solves and approves the draft. Only an organization owner performs the final publish, and only through a single verification gate. A released lesson receives an Ed25519 ProofCard bound to its exact hash and every computed check.

Learners then practice against a synchronized motion model, staged hints, and server-side grading — and can challenge any claim. A supported challenge withdraws the claim, drops its planning weight to zero, rebuilds the dependent rationale, and revokes the stale cached projection in front of the learner. Every account, course, draft, publication, attempt, challenge, and audit event is stored durably in SQLite and survives restart.

How it works

ChalkTrace separates proposal from authority. A candidate enters a strict typed contract; each check evaluates the exact candidate hash and records its verifier identity, version, and evidence digest. publish_candidate is the only publication boundary — missing, duplicate, stale, indeterminate, failed, or tampered evidence blocks delivery. The engineering-mechanics verifier runs 2,048 generated velocity/acceleration cases in Python, mirrored by the TypeScript implementation that drives the learner simulation, and both hashes are part of the correctness evidence. On a committed 50-candidate corpus with seeded defects, all 30 defective candidates are blocked and all 20 clean candidates publish — zero false publishes, zero false blocks, deterministically recomputable by the release gate.

The product moment

A learner challenges an assessment-frequency claim. ChalkTrace checks the cited source, agrees the claim is unsupported, withdraws it, zeroes its planning influence, rebuilds the rationale, and revokes the stale projection — and rebuilds her path in front of her. Trust is an interaction, not a disclaimer.

How I used GPT-5.6 and Codex

GPT-5.6 planned the product — researching the problem, shaping the proof-carrying-learning thesis, and mapping risk — and has a runtime role as the course architect behind chalktrace ingest --propose, drafting candidates from content-addressed spans through a narrow, fail-closed provider boundary. Every draft lands quarantined with a hash-bound proposal trace, and still has to pass all eight checks and educator review before it can publish: the product's own thesis applied to its own model. Codex implemented and verified the repository — the contracts, trust boundary, cryptographic and verification kernels, adaptive policy, the durable studio, the web product, and the tests.

The core /feedback IDs are:

• 019f5fcb-5f69-7892-a338-08c0c7c62573
• 019f6020-605c-71a3-848d-8aabf1385f62
• 019f6097-19af-7292-976c-0182284223d4

Human decisions

The entrant chose the education theme, pushed back on a broad generate-everything implementation strategy, required one complete vertical slice, made ProofCards plus the challenge interaction the product signature, and kept consequential authority in deterministic code.

Challenges

The hardest engineering problem was making every consequential transition evidence-preserving. A rebuilt candidate must invalidate old checks. A helpful hint must not expose the answer too early. An adaptive estimate must show its uncertainty. A withdrawn claim must stop influencing future plans. A cached public projection must not outlive the evidence that justified it.

The solution was to make evidence part of the product state: immutable source bindings, candidate hashes, verifier hashes, signatures, reason digests, dependency changes, state versions, and cache revocation paths are all inspectable.

Accomplishments

• A polished non-chat learning experience where the trust architecture is visible and usable.
• Seven computed policy checks with exact-hash evidence and a fail-closed publication gate.
• A 2,048-case executable verifier tied to the interactive artifact.
• Real public-domain course material released through the same proof pipeline as the synthetic fixture.
• A measured gate yield — 30/30 seeded defects blocked, 20/20 clean candidates published — that the release gate recomputes deterministically.

• A working ingestion path from any course file to a gated, quarantined candidate.
• Trusted ProofCard attestations and session-signed adaptive DecisionCards.
• A challenge that causally changes the future learning path, and an Instructor view that shows the operations side of that loop.
• A private-answer boundary covered by tests and public-projection scans.
• A reproducible release with typing, lint, backend/web tests, SBOM, dependency audit, security policy, and hardened container.

What I learned

Trust becomes much more compelling when it is an interaction rather than a disclaimer. The strongest moment is not showing that a check passed; it is letting a learner disprove a system claim and watching every dependent decision update.

I also learned that model agreement is not publication authority. Typed contracts, executable domain rules, adversarial cases, source hashes, accessibility checks, and cryptographic receipts create complementary evidence that can be inspected and falsified.

The sharpest lesson came from turning the thesis on my own work. A system that invents the misconceptions it claims to find is grading its own homework, exactly the failure it was built to prevent. So every misconception ChalkTrace diagnoses now names a documented misconception family from peer-reviewed physics-education research — Trowbridge & McDermott (1981) for direction/sign and Δv/Δt errors, Beichner (1994) for graph-as-picture and slope/height confusion — and the release gate fails if any diagnosable misconception lacks a resolvable DOI. Those papers document the misconception family, not my items, and the product states that scope on screen rather than borrowing their authority.

What’s next

Richer ingestion adapters and new subject domains, each earning release authority through the same verifier-qualification recipe physics proves end to end. The first adoption target is an engineering or science course team running high-enrollment assessment practice; the concrete two-week rollout is written up in docs/pilot-playbook.md. It uses only surfaces that ship in this repository.

Built With

• codex
• ed25519
• fastapi
• mcp
• python
• react
• typescript
• web-crypto