PARALLAX SPATIAL MCP

Inspiration

PARALLAX started from a simple problem: autonomous agents can call tools, but in spatial environments that is not enough. If an agent moves, rotates, or modifies something in a 3D world, we need a way to know what it was allowed to do, what actually changed, and whether the result matched the objective.

That led to the core loop:

Observe → Act → See Again → Verify → Receipt

The goal was to make spatial agency more accountable instead of treating a tool call as proof that a task succeeded.

What it does

PARALLAX is a Spatial MCP for autonomous agents. It turns 3D environments into governed workspaces where an agent can:

• inspect a scene
• discover bounded capabilities
• request authorized spatial actions
• execute allowed changes
• re-observe the resulting state
• verify the outcome
• produce an execution receipt

Restricted actions can be denied without changing the protected scene state.

Our WebMCP demo uses a virtual production studio. The agent is given the objective of preparing the studio for an interview while operating inside defined permissions.

How we built it

PARALLAX was built with WebMCP, TypeScript, React, Three.js, React Three Fiber, Vite, GLTF assets, and OpenAI Codex, with ChatGPT and Hermes assisting with development, verification, documentation, and submission preparation.

The application separates the visual 3D scene from the authoritative logical state. Scene objects expose spatial capabilities, actions pass through governance checks, the environment is re-observed after execution, and verification determines whether the objective was satisfied before a receipt is shown.

We also built explicit denial behavior so a restricted mutation can fail closed rather than silently modifying the scene.

Challenges we faced

The hardest challenge was making the execution loop understandable visually, not just technically correct.

Early versions used proxy geometry that made real scene changes difficult to recognize. We had to improve object presentation, before/after comparison, labels, camera framing, video backgrounds, and the relationship between the 3D environment and the governance system.

Deployment parity was another major challenge. A local build could look correct while a production deployment could lose assets, motion, layout, or visual hierarchy. That pushed us to add stronger design and production-parity checks.

We also had to be disciplined about claims. A generated visualization is not the same thing as a verified result, so throughout the project we used the rule:

Generated! = Verified

What we learned

We learned that agentic spatial systems need more than tool access. They need an execution contract.

The important part is not simply whether an agent can manipulate a 3D environment. It is whether the system can clearly answer:

What did the agent observe?
What was it authorized to do?
What changed?
Did the change satisfy the objective?
What evidence remains afterward?

PARALLAX is our attempt to make those questions first-class parts of spatial agent infrastructure.

Built With

• ai-agents
• codex
• css3
• gltf
• html5
• javascript
• openai
• three.js
• typescript
• vite
• webmcp