Teach

Inspiration

Some of the most valuable workflows are easier to demonstrate than to describe. Turning a real desktop process into a precise automation specification normally means pausing the work, writing every step down, and hoping the result captures the decisions that actually matter.

Teach lets a person show Codex the work once, inspect what it learned, and keep the result as a reusable skill.

What it does

Teach is an open-source, cross-platform Codex plugin for learning human-demonstrated workflows.

The user can optionally name the workflow, then explicitly authorizes a visible recording. Teach records the bounded demonstration, extracts representative frames, and asks Codex to produce a structured process model: the goal, exact output contract, inputs, software, duration, ordered steps, decision points, risks, verification criteria, and replayability.

The result is never silently automated. The user can review and edit the draft, ask for alternative methods only when they preserve the exact same output, and publish the approved process as a normal portable Codex skill.

How it works

The installable plugin combines:

a $teach orchestration skill;
an MCP Apps interface embedded directly in Codex;
a local MCP server;
native recorder adapters for GNOME Wayland, macOS, and Windows 11;
a file-backed session store using atomic JSON and append-only JSONL;
FFmpeg validation and bounded frame extraction;
schema-constrained analysis through sandboxed codex exec;
deterministic capability and policy checks before publishing.

The packaged plugin includes standalone platform runtimes, so judges can install it without rebuilding the source or installing npm dependencies.

How Codex and GPT-5.6 were used

Codex was the primary development partner throughout the project: it helped analyze the Corridor reference codebase, define the consent and privacy boundary, implement the plugin and MCP Apps UI, build the cross-platform recording adapters, write the deterministic evals, diagnose real installed-plugin failures, and prepare the public release.

GPT-5.6 supplied the core reasoning used during Build Week and the demonstrated structured-analysis path. Teach asks the configured Codex model to turn bounded visual evidence into a strict process schema; deterministic code then validates state transitions, replayability evidence, generated skill structure, and publishing paths. The model cannot silently record, publish, or execute a workflow.

The repository preserves the prompt history and engineering decisions in its prompt library and devlog.

Why it is different

OpenAI Record & Replay validates the interaction pattern. Teach explores a different product boundary:

Open source: Apache-2.0 implementation and inspectable artifacts.
Cross-platform: Linux, macOS, and Windows 11 support, with especially strong GNOME Wayland handling.
More than replay: rich process labelisation, output contracts, risks, blockers, and verification criteria.
Capability-honest: workflows are labeled replayable, assist-only, unsupported, or unknown from available evidence.
Reviewable: the learned process can be edited before it becomes a skill.
Portable: the output is a standard Codex skill rather than a hidden pixel macro.
Privacy-led: capture is visible, scoped, pausable, short-lived, and stored locally.

Teach is independently implemented and is not affiliated with or a replacement for OpenAI Record & Replay.

Challenges and lessons

The hardest part was not recording pixels; it was building a reliable consent-to-skill product across real desktop environments. Early installed-plugin tests exposed missing native buttons, Wayland session failures, MCP parameter mismatches, stale temporary-directory warnings, long analysis with no progress feedback, and host-specific UI action timing. Each failure became a regression test or a product improvement.

The result is a working product experience with native controls, visible recording state, animated analysis progress, editable review, deterministic evaluation, and three-OS CI.

Try it

Repository: https://github.com/Zacxxx/teach

Demo: https://youtu.be/1xKRY7CZj8A

Deterministic no-credentials judge path:

git clone https://github.com/Zacxxx/teach.git
cd teach
bun install --frozen-lockfile
make check
make demo

The demo creates a synthetic recording, analyzes it with the fixture analyzer, and publishes a temporary skill. It does not record the judge's desktop and does not require an OpenAI API key.