CargoPilot

Inspiration

Portfolio builders are visual, but collaborating with an AI in one often means describing a layout from memory or exchanging disconnected screenshots. CargoPilot explores a more explicit shared reference: the creator chooses a signed-in Cargo page, the agent receives its current visible context, and both know when that context is too old to trust.

What it does

CargoPilot connects a public WebMCP companion to a private local bridge. It exposes a screenshot, bounded semantic context, route and page information, capture time, and freshness state for a selected Cargo screen. The same context is available through a standard stdio MCP server.

There are two distinct connection paths. A Chrome extension lets the user click Share with Codex on an already signed-in Cargo tab. In Codex/ChatGPT's in-app browser, a browser-capable host agent can instead make an explicitly authorized capture and relay it using request_cargo_context and attach_cargo_context. Extensions cannot be installed inside that in-app browser; the hosted companion cannot independently inspect another tab or borrow its authentication.

The shared canvas is a read-only snapshot, not a second Cargo editor. Captures expire after 15 seconds. The UI marks stale and unavailable states, and context tools do not silently switch to another browser. The creator scrolls or edits in Cargo and asks the agent for a new capture when needed.

The Chrome extension also implements a separate time-limited approval path for two fixed recipes: exact text replacement and native Backdrop selection, with route and postcondition checks. Read-only sharing does not authorize these actions. The final live demonstration deliberately exercises only read-only context; it does not claim to demonstrate those editing recipes, asset uploads, arbitrary template editing, or publishing.

Why WebMCP

WebMCP turns the companion's collaboration boundary into named, typed tools. An agent can ask for workspace, screen, or editor context through document.modelContext.registerTool(...) instead of guessing what a screenshot or inactive canvas means. Inputs are validated, freshness is explicit, and inspection is separated from mutation authority.

The creator can discuss a specific page while the agent receives the relevant visual evidence and knows which site, page and capture it belongs to. The Chrome pointing interaction additionally shares viewport coordinates and a bounded accessible label. This reduces repeated manual explanation without treating unrestricted browser control as the default.

How we built it

• Vite and TypeScript companion, hosted on Vercel, registering browser-native WebMCP tools.
• Local Express service bound to 127.0.0.1, with Zod input validation, short-lived context, audit state, and a standard MCP server.
• Manifest V3 Chrome extension with explicit selected-tab sharing and a separate approved fixed-action path.
• Browser-host attachment for the in-app browser, using actual screenshot capture and bounded visible context without moving cookies or passwords between browsers.

Codex assisted with implementation, debugging and testing. Remotion assembles real browser captures and measured capture sequences into the demo. The narration is AI-generated; the conversation section is explicitly labelled as a replay of our actual exchange, not a recording of the Codex interface.

Validation

The 4 September 2026 live pass retrieved a real authenticated Cargo dashboard and an editor page through the deployed WebMCP companion and an actual stdio MCP client. Both returned the relevant current screen and screenshot. Asking for editor context on the dashboard correctly failed. Expiry produced an explicit stale response and visible UI label. Losing the attachment removed the old canvas image and left unsupported typing controls disabled.

There are 64 passing automated tests, TypeScript checks pass, the hosted build passes, and extension 0.4.1 packaging and response-parsing checks pass. The installed Chrome extension's attach and mutation flow was not re-tested in this final live pass. No live Cargo mutation or publishing tool was called during validation or filming.

Challenges and lessons

Browser-session isolation was the central challenge: a hosted page cannot safely reuse an unrelated tab's login. The viable design is explicit sharing through a permitted Chrome extension or a browser-capable host, not a hidden second browser profile. Freshness also matters: an old screenshot must not be presented as live editor truth.

The lesson is that useful agent collaboration needs a trustworthy reference and a clear scope, not just many tools. CargoPilot is a local-assisted prototype, not a cloud-only mobile service or an autonomous general-purpose Cargo editor.

What's next

Package the local bridge as a signed installer, improve guided reconnection, and expand supported editing recipes only when each has deterministic verification. A first-party continuous host attachment API could reduce repeated captures without weakening consent.

Built With

• chrome
• express.js
• model-context-protocol
• playwright
• typescript
• vercel
• vite
• vitest
• webmcp
• zod