AGENT GATE

Inspiration

AI agents can act fast, but high-consequence actions still need human judgment. Traditional confirmation screens compress people, files, permissions, and context into a flat list. That works for simple choices, but it becomes harder when the right answer depends on relationships and exceptions.

AGENT GATE explores a different interaction model: make the agent's proposed consequence spatial and editable before execution.

What it does

The judge-facing scenario begins with an AI agent proposing to send Q3_Report.pdf to four recipients.

Two recipients are clear internal collaborators. Two are marked YELLOW because they require human judgment rather than automatic rejection.

The user simply aims at a recipient card to reveal its context. Pinching is reserved for editing: grab an unauthorized recipient, drag that consequence into HOLD FROM SEND, and pinch APPROVE.

In the example, External C is an active audit contact with an NDA and a current business need, so the user keeps it. External D is a former employee using a personal mailbox with no current business need, so the user removes that recipient from the proposed action.

The pending action updates immediately. After approval, AGENT GATE confirms the corrected future state: three recipients continue, one is held, and the edited plan is returned to the agent.

Why spatial

The core interaction is not a chatbot in VR. The proposed action, its participants, uncertainty, and the effect of removing one element remain visible at the same time.

Spatial persistence makes the intervention concrete: instead of toggling an abstract checkbox, the user edits the future state the agent is about to create.

How we built it

AGENT GATE is built with Meta IWSDK / WebXR, Three.js, and TypeScript.

It uses hand tracking, aim-based context inspection, RayInteractable, and DistanceGrabbable for direct spatial editing.

The experience is designed for seated/stationary use. Its core flow is fully hands-first and can be completed end-to-end without pairing a controller.

During development, automated GitHub Actions tests validated the decision model, production build, spatial edit flow, hands-only interaction, and the judge-facing experience using the IWER Meta Quest emulator.

Challenges we ran into

The biggest challenge was making the experience understandable in the first few seconds. Early versions made the decision rule difficult to read and mixed inspection with editing.

We separated the interactions so that aiming reveals context while pinching is reserved for editing, improved the spatial layout, and made the HOLD zone and decision rule immediately visible.

What we learned

Spatial computing becomes valuable when space changes the decision itself, not when a 2D interface is simply placed in VR.

The strongest version of AGENT GATE emerged when we focused on one mechanic: letting a human directly edit the consequence of an AI agent's proposed action.

What's next

The same consequence-editing model could extend to file sharing, access changes, scheduling, purchasing, and other high-impact agent actions, with organization policies and audit trails.

Built With

• actions
• github
• iwer
• iwsdk
• meta
• three.js
• typescript
• vite
• webxr