OpenScene Studio

A trainer asks ChatGPT to add a missing exchange to the video lesson already open. OpenScene waits for the learner before releasing the recorded answer.

Comment

Inspiration

A video lesson about changing trains in Germany includes an announcement that the current train ends here and the connecting train leaves from railway platform two. It never teaches a passenger who cannot use stairs how to ask a station employee for the lift. The learner does not know the German words.

The trainer can create a separate exercise or re-edit the recording. A separate exercise removes the practice from the original scene. Re-editing repeats the production work.

OpenScene Studio lets the trainer add that missing exchange to the lesson already open in the browser.

What it does

The trainer tells ChatGPT what the learner needs. Through WebMCP, ChatGPT reads the current OpenScene project and adds a practice path using the response clip, route board, and timing already approved in that project.

OpenScene pauses the lesson before the recorded station employee's answer. The learner chooses Wo ist der Aufzug zu Gleis zwei? Only then does the page release the response clip and show the LIFT → PLATFORM 2 board. The trainer can keep the path or undo it.

This gives each person a clear role:

  • The trainer defines the learner need and approves the lesson change.
  • ChatGPT maps the request to the open project's branch fields.
  • OpenScene supplies the approved German line, recorded response, board, timing, and revision history.
  • The learner performs the missing turn before seeing the answer.

Why WebMCP

An ordinary chat can suggest the German sentence, but it cannot edit the OpenScene lesson already open in the browser because it cannot access that lesson's cue, trainer-approved response clip, pause point, or revision. Through WebMCP, OpenScene exposes those controls as narrow page-owned tools.

OpenScene registers six narrow tools through document.modelContext.registerTool(...):

  • openscene_inspect_project
  • openscene_configure_project
  • openscene_propose_branch
  • openscene_update_branch
  • openscene_preview_branch
  • openscene_undo_last_edit

ChatGPT can describe the learner need, supply the learner-facing line, and select a page-approved response pack. It cannot inject response words, board text, media paths, or answer timing. Those remain page-owned. The demonstrated writes carry the expected page revision, so a stale request fails without changing the project.

That shared state lets one plain-language request coordinate the lesson cue, learner line, pause, recorded response, preview, and undoable project revision on the page the trainer is already using.

How we built it

OpenScene Studio is a React and TypeScript application built with Vinext and hosted on ChatGPT Sites. A revisioned StudioBus owns the project state and validates every WebMCP read and write. Playwright covers the visible browser flow, while deterministic tests cover schemas, stale-revision handling, unknown-field rejection, learner gating, trainer approval, media decoding, accessibility, and the production build.

The public release passes 137 deterministic tests, 45 browser checks in development, and the same 45 checks against the production server. The local release package also passes five private evidence checks that are intentionally excluded from the public source archive.

The demo displays the trainer's request on a labeled editorial card. The next split view shows the authentic privacy-cropped ChatGPT and OpenScene browser capture. Four jump cuts enlarge the recorded tool names, inputs, results, and page revisions while removing model waiting time. The learner click and the six-tool implementation proof appear on separate, clearly labeled frames.

Challenges and lessons

The main design decision was what the agent could change. Letting it write the station answer or choose arbitrary media would weaken the trainer's authority. We kept those elements in approved response packs and exposed only the learner-specific edit through WebMCP.

The second challenge was proving the human turn. The response cannot start as soon as ChatGPT edits the project. The page must pause, wait for the learner's German line, then release the answer. That sequence turned the idea from an editor automation into a real practice exchange.

WebMCP works here because ChatGPT interprets the trainer's request while OpenScene controls the lesson content, project state, and the decisions left to the learner and trainer.

Potential impact

For the learner, this turns a passive scene into the exact exchange they need to practise before a trip. For the trainer, it adds that exchange inside the original lesson instead of creating a separate exercise or editing the whole video again.

The same page-owned pattern could support other approved practice paths, including a caption-first explanation, a slower language turn, or a shorter sequence. This prototype proves one mobility-and-language path. It does not yet measure learning or accessibility outcomes.

Scope

The demonstrated station lesson is fictional and intended for at-home language practice. It provides no live travel data, wayfinding, speech recognition, character perception, or measured learning outcome. The scene-partner imagery is synthetic and the authored response clips are silent.

Built With

Share this project:

Updates

Submission history