Physical AI WebMCP Command Center

An Agent-Native Control Layer for Autonomous Warehouses.

Comment

Inspiration

Warehouse software is usually designed for humans clicking through multiple screens. Browser agents can interact with those interfaces, but they must infer operational meaning from pixels and DOM structure. We wanted the web itself to expose a safe, typed contract shared by humans, AI agents, and autonomous machines.

What it does

Physical AI WebMCP Command Center is an agent-native warehouse control layer built around a deterministic digital twin.

A user gives one intent: move pallet P-104 from INBOUND-01 to RACK-A12. Through seven WebMCP tools, the Agent observes warehouse state, inspects the pallet, plans a safe route, selects AGV-03, and creates an approval-gated proposal.

A human must approve before physical movement begins. During execution, AGV-03 encounters a blocked aisle at N07, stops safely, replans through N08 and N11, and completes the mission.

The safety engine validates destination, occupancy, vehicle availability, battery reserve, route safety, approval state, communication heartbeat (SAFE-11), and traffic reservations (SAFE-12).

Every WebMCP call, result, and measured latency is displayed in the interface. The observable workflow is:

OBSERVE → PLAN → VALIDATE → APPROVE → EXECUTE → RECOVER

How we built it

The application uses React and TypeScript and runs on a public HTTPS deployment.

A shared deterministic state engine powers both the manual UI and WebMCP interface. Dijkstra routing produces the 41.6 m primary route and the 49.4 m recovery route. The simulator controls AGV movement, fault injection, safe stopping, replanning, resuming, and mission completion.

Exactly seven WebMCP tools are registered with strict JSON Schema, read/write annotations, cancellation support, and production discovery. Runtime instrumentation measures planner stages, tool latency, route length, approvals, replans, mission success, and Human vs Agent performance.

Challenges

The main challenge was preserving a trustworthy boundary between AI intent and physical execution.

Planning must never start a mission. Creating a proposal must never imply approval. An unapproved mission cannot move. A route blockage must stop the AGV before replanning. Communication timeouts and conflicting traffic reservations must deterministically prevent unsafe movement.

We also ensured that all displayed traces and metrics come from real runtime events rather than scripted presentation data.

Accomplishments

  • Seven discoverable WebMCP tools with strict schemas
  • Human approval that cannot be bypassed by an Agent tool
  • Deterministic Dijkstra planning and blockage recovery
  • SAFE-11 communication-timeout enforcement
  • SAFE-12 traffic-reservation conflict prevention
  • Visible WebMCP and planning traces with real latency
  • Complete reset of missions, proposals, traces, metrics, timers, and faults
  • Human vs Agent benchmark using the same operational task
  • Public HTTPS deployment with a verified 7/7 Agent E2E run
  • No account, API key, database, broker, or physical hardware required

What we learned

Agent-native UX is not about removing human control. The strongest design separates responsibilities:

The model expresses intent. WebMCP defines how it asks. Deterministic software decides what is allowed. A human authorizes high-impact action. The physical system executes.

What's next

The same WebMCP interface could be connected to real fleet-management systems. Approved missions could be translated into VDA5050, MQTT, PLC, or vendor API commands while retaining deterministic safety policies, audit traces, and human authorization.

Built With

Share this project:

Updates