CueMend

A WebMCP rehearsal room where humans protect dramatic beats, agents exhaustively repair caption timing, and every committed WebVTT track is revision-bound and verifiable.

Comment

Inspiration

Caption timing is both mechanical and artistic. Software can detect overlapping captions, reading-rate problems, missing speakers, and visual collisions, but it cannot decide whether an audience laugh, lighting reveal, silence, or actor's breath carries artistic meaning.

CueMend explores a better division of responsibility: let the computer perform exhaustive, inspectable checking while the stage manager retains artistic judgment and final authority. It is designed around small theatres, school productions, and live-event teams that need systematic caption checks without handing the meaning of a performance to an optimizer.

What it does

CueMend opens a fictional 32-second rehearsal containing ten caption cues and seven independently reproducible timing issues.

Through WebMCP, a compatible browser agent can:

  • inspect the exact rehearsal state and revision;
  • audit caption overlap, minimum gaps, duration, reading rate, speakers, protected beats, and reserved screen regions;
  • ask CueMend's deterministic browser engine to evaluate all 3^8 = 6,561 bounded candidate tracks;
  • stage up to three zero-conflict alternatives without changing the active captions;
  • preview or discard an exact proposal;
  • commit only the plan explicitly selected and approved by a human;
  • run a post-commit deterministic re-audit;
  • export editable WebVTT plus a SHA-256-bound evidence certificate.

The 6,561 figure is the exact search space of this demo: eight adjustable cues, each with three authored repair variants. It is not a universal theatre number, and the engine never invents or rewrites dialogue.

The signature interaction begins when the cheapest proposal crosses Jon's optional final breath. The stage manager decides that the breath matters and protects it in the visible UI. CueMend increments the workspace revision, reports an additional conflict, and makes the old proposal stale. An attempt to use the old revision receives a structured STALE_REVISION refusal. The agent must inspect the new state and replan under the human's updated constraint.

After the human selects and approves one exact plan, a temporary commit capability appears. The commit is bound to the approved plan, current revision, protected beats, profile, and digest. After it is used, mutation tools retire. The finished rehearsal reaches revision 3 with zero conflicts under the named demo profile.

Why WebMCP matters

CueMend remains usable as an ordinary web application, but WebMCP turns it into a shared, state-aware collaboration surface for humans and agents.

Without WebMCP, a browser agent would otherwise have to infer domain state from the DOM, automate brittle coordinates, or depend on a CueMend-specific integration. With WebMCP, the page exposes domain-level tools over the same live state the person sees: exact cue IDs, locks, revisions, alternatives, structured errors, and safe next actions.

Tool availability follows the real authority state:

  • a fresh workspace exposes inspect, audit, and stage tools;
  • a current proposal adds preview and discard;
  • a stale proposal cannot be previewed or committed;
  • the commit tool appears only after a person approves the exact artifact;
  • after commit, mutation capabilities disappear and verification becomes available.

This means the human does not merely click a cosmetic approval button. A human decision changes the optimization problem, invalidates previous agent work, and changes which capabilities the page exposes to the agent.

Without WebMCP, CueMend is a caption-checking and repair application. With WebMCP, it becomes a standardized, tab-bound workspace where repetitive work can be delegated to a compatible browser agent without surrendering artistic decisions.

How we built it

CueMend is a dependency-free static application built with semantic HTML, responsive CSS, and native JavaScript ES modules.

A deterministic engine audits interval, duration, reading-rate, speaker, protected-beat, and reserved-region constraints. It enumerates the exact 6,561-candidate demo search space and ranks valid tracks by:

  1. number of changed cues;
  2. total timing movement;
  3. semantic changes such as speaker or position;
  4. a stable lexical key for deterministic output.

A shared command layer owns revision validation, proposal and plan digests, human-only operations, idempotent request IDs, session-bound commit receipts, and structured error recovery. The ordinary UI and WebMCP handlers call this same command layer and update the same visible artifact.

The page registers tools with document.modelContext.registerTool. Registrations are retired with AbortSignal when the capability inventory changes. Every consequential handler repeats runtime validation because schemas and annotations are not treated as security boundaries.

Web Crypto produces SHA-256 digests, and Node's built-in test runner verifies the engine, authority model, and dynamic tool lifecycle without third-party packages.

Challenges we ran into

The hardest design problem was making human approval substantive instead of decorative. The person needed to change the actual optimization problem by protecting an artistic beat, while the system needed to invalidate an in-flight proposal reliably and explain the recovery path clearly.

The evolving WebMCP API also required a careful distinction between registration-lifetime cancellation and execution cancellation. CueMend registers or retires tools only when the available capability set changes, while revalidating all important state immediately before mutation.

Another challenge was keeping every numerical and accessibility-related claim auditable. CueMend names its exact demo profile, reports the number of candidates evaluated, preserves dialogue, and displays its limitations instead of implying universal caption compliance.

Accomplishments that we're proud of

  • A human decision causes a real agent replan, not a cosmetic approval step.
  • The deterministic engine evaluates every candidate in its disclosed finite search space.
  • A stale proposal cannot be previewed, approved, or committed after a human changes a protected beat.
  • Commit is bound to the selected plan, current locks, profile, revision, and SHA-256 digest.
  • One request ID can apply a change at most once and returns the same receipt if replayed.
  • The final artifact is editable WebVTT rather than a result trapped inside a chat.
  • The visible application remains fully usable when WebMCP is unavailable.
  • The complete native WebMCP journey was repeated three times from a clean reset in Chrome 151, with ten native calls per run and a genuine stale-revision refusal each time.

What we learned

The most useful WebMCP tools are small domain verbs over live, visible state. Tool count matters less than clear preconditions, explicit postconditions, recoverable errors, and meaningful authority boundaries.

Dynamic tool registration becomes especially valuable when it mirrors real authority: the agent should not even receive a commit capability until a person has approved the exact artifact.

We also learned that a strong agent experience does not require hiding the application behind chat. Human and agent interfaces can operate on the same state, with every transition remaining visible, reversible before approval, and verifiable afterward.

What's next

Future versions could add real WebVTT import, user-authored production profiles, venue-specific safe areas, rehearsal-note exchange, and integration with existing caption playback systems.

Those extensions would preserve the same principle: automation proposes and proves mechanical changes, while production teams retain control over artistic intent.

Important limitations

All dialogue and production details in the built-in rehearsal are fictional and rights-safe.

The CueMend Demo Theatre Profile v1 is a transparent demonstration profile, not a universal WCAG, broadcast, venue, legal, accessibility, or artistic standard. CueMend does not claim to certify accessibility or replace professional caption review. No generative model runs inside the submitted application; its repair engine is deterministic and local to the browser.

Built With

Share this project:

Updates

Submission history