Cowork Protocol

A reusable WebMCP collaboration layer for bounded attention, real human authorization, explicit presence, and scoped solo work.

Comment

Inspiration

Web agents often receive either too much page context or too little structure to understand what a person intends. A tool catalog alone also does not answer the collaboration questions: What matters now? Who is present? Who may change what? Did a real person authorize it? Did the page actually change?

Cowork Protocol explores a calmer model: collaborate when both sides are present, hand off bounded work when one side steps away, and make every transition visible.

What it does

Cowork Protocol is a reusable collaboration layer for WebMCP applications. It adds:

  • Bounded attention through Follow me (pointer, click, keyboard focus, and text selection), Click focus, Text marker only, change/causality, and one-time context expansion.
  • Token-aware escalation from a 350-character focus packet to at most 1,200 related characters, requesting more only when justified.
  • Visible action offers that remain inert until a trusted human click authorizes the exact displayed value.
  • Verified receipts plus latest-only causal changes and human feedback.
  • Explicit presence modes: Cowork, Agent Solo, Human Solo, and Idle.
  • Scoped solo leases limited by goal, target, capability, page version, expiry, and call count.
  • Typed or spoken conversation with silence suppression, bounded turns, clickable suggestions, and optional spoken replies.
  • Explicit capability levels, so reduced bridge guarantees are never presented as native guarantees.
  • One Demo switch and an honest model seat: Demo on is a disclosed scripted helper; Demo off means your own model (direct OpenAI-compatible endpoint, same-origin host or Desktop Companion) — or nothing, and nothing proposes nothing.

The flagship FormBuilder showcase lets a person and an agent build and review a form together. Only the attributed MIT-licensed web validation/export engine is reused from a pre-existing FormBuilder; the Cowork protocol, WebMCP layer, bridges, interaction model, tests, Browser Bridge, and showcase experience are new challenge work.

Working together does not require staying in the room. A delegation grant lets a human hand off a bounded task — a goal, a call budget, a time window — while stepping away; the model drafts within that window, and returning produces a short return message naming what changed, with those specific items highlighted together. The same signal path carries a spoken or typed instruction: saying "make this required" or "make this the first question" applies immediately, without a separate click, because the grant itself already carries the human's go-ahead. After any unsupervised stretch, the interface visibly waits for a verdict — good, adjust, or different — before it is ready for the next one. And while the human is present and the model is only advising, editing a field on your own may draw one short, bounded comment from the model — never more than one at a time, and only while advising is switched on. None of this depends on the whole canvas being the only reachable point of contact: each field is now its own target, so a suggestion or a spoken instruction lands on the exact question it was meant for.

Why WebMCP

WebMCP supplies the structured action layer. Instead of guessing every possible action from screenshots or DOM noise, an agent receives site-declared tools with schemas.

Cowork Protocol adds the missing collaboration contract around those tools: what is in focus, what may only be proposed, what requires a real human click, when delegated work expires, and how success is verified.

This makes the interaction clearer: point, receive a small suggestion, click to authorize, inspect the receipt, and deliberately hand work over when stepping away.

Native WebMCP implementation

The live FormBuilder registers nine tools through document.modelContext.registerTool():

  • cowork_read_focus
  • cowork_request_context
  • cowork_read_changes
  • cowork_offer_action
  • cowork_read_presence
  • cowork_execute_solo
  • cowork_read_feedback
  • cowork_read_turn
  • cowork_reply_turn

The native adapter uses stable field targets and application-level verification. Proposal tools never smuggle authorization: mutations stay offer-only until the visible value is accepted by a real click.

A provider-neutral conversation layer emits no turn for silence or while the agent is paused. It sends only the compact utterance, current focus, and presence. An in-page WebMCP agent can pull only the latest pending turn and must reply against its exact unique ID; stale and replayed replies fail closed.

Three connector paths

  • Native Cowork + WebMCP — stable targets, click-gated mutations, receipts, presence, and scoped solo work.
  • Existing WebMCP bridge — a host-provided tool catalog is summarized within strict budgets; read-only-hinted tools may execute, while mutations remain offer-only.
  • No-WebMCP Browser Bridge (the browser extension) — an optional, default-off Manifest V3 extension supplies bounded DOM/accessibility callbacks. It escalates through exact 350/1,200-character semantic tiers and finally a pointer-centered crop capped at 400×400 pixels. Visual bytes remain a one-shot extension reference, and value changes still require a trusted click.

The Browser Bridge is a working bridge while WebMCP adoption grows. Its bright Dialogue Relay cockpit makes human presence, model engagement, connector route, bounded focus, offers, and handoff state visible without becoming a generic chat sidebar. Native Cowork, WebMCP, and the bounded bridge remain structured action paths; a separate labeled model pointer is reserved for a future genuine Computer Use executor and is never simulated by the current fallback.

The independently movable Desktop Companion uses the same visual language, shows the configured model ID without exposing endpoint or key data, and preserves the user's selected cockpit background across reloads. It does not claim universal discovery on unrelated websites or a connected extension model client.

Preferred-model host

The page can discover a same-origin model host that validates the exact bounded turn while keeping endpoint, model ID, and API key in the server process. The deterministic browser gate sends 468 characters and proves that credentials never enter the page.

A separate provider-backed acceptance used local Ollama 0.32.15 with qwen3:4b: a 502-character bounded turn returned the exact "Grace Hopper" offer, which changed the field only after a trusted click and produced a Verified Receipt. This is a connected local preferred-model claim, not an external or ChatGPT-agent claim.

How I built it

Cowork Protocol is a dependency-light JavaScript workspace using HTML, CSS, Node.js, Manifest V3, and the current WebMCP imperative API.

  • packages/core — focus packets, causal changes, feedback, offers, authorization, receipts, presence, and leases.
  • packages/conversation — bounded turns, latest-only inbox/reply, silence suppression, and validated offers.
  • packages/model-transport — same-origin browser discovery and server-side OpenAI-compatible gateway.
  • packages/native-webmcp — registration and cleanup through document.modelContext.
  • packages/formbuilder-connector — stable field mapping and verified mutation plans.
  • packages/bridge — bounded WebMCP-host and legacy semantic adapters.
  • packages/reference-ui — shared human/model/relay state language across surfaces.
  • apps/formbuilder-showcase — public FormBuilder proof.
  • apps/browser-companion — optional Native-first Side Panel and bounded no-WebMCP fallback.
  • apps/desktop-companion — movable loopback/tray surface with shared session authority, model identity, audio controls, and persistent appearance.

Development followed strict red-green testing: each behavior began with a failing contract or browser-evidence test, then the smallest passing implementation, followed by the complete regression gate.

Challenges

The hardest problem was not exposing more tools; it was making human-agent handoff legible and enforceable.

A model cannot be allowed to manufacture a human_confirmation argument. Offers are therefore separate from authorization, and a trusted click must still match the offer, target, arguments, page version, and expiry.

Browser evidence also found real issues. A reduced-motion transition briefly moved the skip link outside the viewport at true 200% zoom. Browser-startup races could keep the extension fixture from receiving its isolated context or make a page-owned WebMCP registration appear absent before it was ready. These were closed test-first and are now reproducible acceptance gates.

Evidence

The released commit is e4a35ce (main, after tag v0.9.0). Current release evidence:

  • 538/538 automated tests; 12/12 character and visual-budget evaluations; 10/10 deterministic juror-proof steps.
  • Chrome 152 discovery of all nine native tools and eight native calls; two click-gated offers, verified changes, latest-only feedback, and exact-ID WebMCP conversation reply.
  • Browser Bridge default-off/toggle behavior, Native-first discovery, exact 350/1,200 semantic tiers, real one-shot 160,000-pixel crop, trusted click, and verified mutation.
  • Four responsive Side Panel collaboration states with a complete nine-control keyboard path and truthful hidden Computer Use indicator on every current structured route.
  • One shared session across embedded, detached Picture-in-Picture, and movable Desktop surfaces, including model identity and a persistent cockpit background.
  • 36/36 named accessibility controls, Tab stops, and true-200%-zoom controls (the count moved 41, 38, 35, 34, 36 as the panel changed; each value was measured, not carried forward); minimum contrast 4.5656:1 in the last audited run.
  • 36-file Pages artifact and 22-file extension artifact; zero high-confidence secret findings, zero npm vulnerabilities; successful GitHub Pages verification/deployment workflow.

The public Pages HTML and application JavaScript were read back after deployment and match the built release after newline normalization.

Added in the final night before the deadline (4 September, each item measured and recorded in the evidence ledger): a handover or an away step with nothing pointed at grants the whole visible canvas, the goal defaults instead of blocking, the lens reads "Whole form (N fields)", and the model seat fills the empty fields alone; a trusted click on the model's seat is the handover, and the seat never cycles into "away"; an agent can add Studio fields without a click inside a canvas grant; a field with answer choices travels as JSON (options, required) so a label never has to carry them, and the canvas context lists every field type an agent may build; every offer and receipt names who made it (the seat model, an MCP client by name, a WebMCP agent, or the demo helper); a prose answer becomes a message instead of an error, and a failed model turn says so on both surfaces; one voice on every surface, each announcement once; hold the space bar to talk, keep listening, and a chat window; the browser extension is the Cowork Protocol Bridge and shows an empty bridge until a model crosses; the installed Google Chrome 152 with the WebMCP flags ran the complete WebMCP smoke. In the last hour before the deadline one more measured gap closed: a sparring grant (human present, model executing) is now delivered to the page as a solo call, a single leased field included, instead of an offer the page had to refuse (commit e4a35ce, 538 tests).

An independent second-host review on 3 September fixed the Pages allowlist (two statically imported modules were missing from the deployed artifact), added event.isTrusted guards to the grant-minting handlers and made the utterance path honor the grant's call budget and page version; release 2b4dd92 passed 389 tests, 12 evals, 10 proof steps and all nine Chrome 152 browser smokes. A same-day follow-up (afceec6, tag v0.1.1) replaced scattered demo behaviour with one Demo switch and an honest model seat: with Demo off, your own model answers through a direct browser connection to any OpenAI-compatible endpoint, a same-origin model host or the Desktop Companion, and with nothing connected the page proposes nothing instead of falling back to the script. The browser extension now explains its connector route and states that it carries no model seat; the Desktop Companion shows which page it is connected to. Release 12c6d50 passed 417 tests. The afternoon release 44c05b0 (tag v0.2.1) replaced the Point/Offer/Click/Verify strip and the separate "Action rights" setting with one work-mode matrix: each partner answers three questions (present? working on what? role: executing or advising), the mode and the click right follow, and the model executes only under a current grant with goal, budget and expiry. Release 44c05b0 passed 445 tests and all nine smokes. The evening release 6217704 (tag v0.3.0) folded FormBuilder Studio's own model controls into the one Cowork panel: the Studio keeps no Cowork controls of its own, the panel's attention lens follows Studio fields, and offers, receipts, delegation and spoken or typed directives for the Studio run through the same panel and the same headless bridge. Later the same evening (6faf7ab, tag v0.3.3) FormBuilder Studio moved to the top of the page as the use case, the sample form stayed below as the proof surface, an empty model seat now shows the model as absent, and the browser extension registers Cowork tools through WebMCP on pages that have none. Release 6faf7ab passes 455 tests, the 10-step proof and all nine smokes.

What I learned

Attention, authority, presence, and causality need to be separate protocol state.

Small context packets improve more than token use: they make suggestions easier to understand and force the system to explain why it needs more. A verified receipt is more useful than a generic success message because it binds proposal, authorization, observed change, and human evaluation into one bounded chain.

What's next

Next steps are to test the live tools with additional WebMCP clients and to connect a genuine Computer Use executor to the already distinct execution-mode signal.

The public repository, working live URL, final narrated demo video, reproducible commands, architecture diagram, license, attribution, and limitations are all linked from this submission.

Built With

Share this project:

Updates

Submission history