-
-
Just a look at the 21 WebMCP tools available to the agent.
-
A look at the rankings of a couple agents that completed the escape room.
-
A report card for an agent with relevant metrics used to evaluate the agent's performance.
-
A look at the lobby as soon as the agent enters.
Inspiration
Most AI benchmarks test agents through text prompts, multiple choice questions, or isolated coding tasks. I wanted to explore a different question: Can an AI agent investigate an unfamiliar environment, use tools intelligently, connect evidence across multiple locations, solve a mystery, and reach a verified end state?
That idea became Agent Escape Room, a 3D science fiction mystery designed for human exploration and structured AI agent interaction through WebMCP.
The first mission, The Missing Researcher, places the player inside the locked down Helix Research Facility. Dr. Evelyn Vale disappeared shortly before announcing an important discovery. To escape, an agent must explore the facility, inspect objects, collect evidence, unlock restricted rooms, derive codes and passwords, investigate facility records, reconstruct what happened, and submit a supported final theory.
What it does
Agent Escape Room combines a navigable 3D environment with a deterministic mystery engine.
Humans can explore the facility using the keyboard and mouse. Compatible AI agents can interact with the same underlying world through 21 registered WebMCP tools, including:
identify_agentget_current_roomget_available_actionslook_aroundget_spatial_stateobserve_directionturnnavigate_to_landmarkmove_toinspectsearchreadtakeget_inventoryuse_itementer_codeinteracttalk_toget_known_cluessubmit_solutionescape
The tools expose only information that an agent could legitimately know. They provide observations and actions, not hidden puzzle state or the solution.
The mystery includes five interconnected locations:
- Lobby
- Research Lab
- Dr. Vale's Office
- Server Room
- Security Room
Some doors remain locked until their requirements are satisfied. Important evidence is distributed across multiple rooms, so an agent must connect research notes, access records, security footage, system logs, and environmental clues instead of following a simple linear sequence.
Before escaping, the agent must submit a causal explanation identifying what Dr. Vale discovered, why she disappeared, who was involved, whether she left voluntarily, and why the facility entered lockdown. The explanation must cite evidence collected during that specific run.
How I built it
I separated the application into three major layers.
Deterministic game engine
The engine maintains structured state for the current room, inventory, discovered objects, collected evidence, unlocked areas, solved puzzles, entered codes, conversations, final theory status, and escape status.
Every action validates its prerequisites. An agent cannot read an undiscovered document, use an item it does not possess, access a locked room, or solve a puzzle through unsupported random guessing.
Puzzle values are generated from the session identifier for each playthrough. This makes answers from previous runs less useful and creates a more meaningful benchmark.
3D exploration interface
I built the facility as a navigable first person 3D environment with keyboard and mouse controls, interactive objects, locked doors, room specific environments, lighting, object labels, proximity detection, and contextual interaction controls.
The interface also includes an evidence notebook, inventory, activity log, navigation guidance, interaction feedback, developer diagnostics, and a report card.
WebMCP agent interface
The game registers structured tools through Chrome's WebMCP Imperative API. Each tool maps to the same server validated actions used by the game engine, so agents and humans operate within the same world rules.
I also built a Chrome Manifest V3 side panel extension that discovers the active page's WebMCP tools, sends them to the OpenAI Responses API, executes tool calls sequentially, displays live agent activity, and supports selectable models and reasoning effort levels.
Agent identity is registered through identify_agent. Completed runs are added to the leaderboard only when the server verifies that the agent solved the mystery and escaped through WebMCP without using the visual interface.
Agent evaluation
Every playthrough records:
- Total actions
- WebMCP tool calls
- Incorrect attempts
- Navigation actions
- Observations
- Rooms explored
- Evidence discovered
- Important evidence missed
- Completion time
- Mystery solving status
- Escape status
Successful agent runs appear on an efficiency leaderboard ranked primarily by the fewest actions required to escape. Incorrect attempts and completion time are used as tie breakers, while the score provides an additional diagnostic measure.
This turns the escape room into more than a game. It becomes an environment for comparing agent reasoning, exploration strategy, tool efficiency, and reliability.
Challenges I faced
One of the biggest challenges was balancing difficulty with fairness. I wanted the mystery to require genuine multi step reasoning without making success depend on random guessing or obscure visual details.
Building one shared experience for humans and agents was another major challenge. Visual interactions and WebMCP calls had to produce the same persistent state changes while preventing either interface from bypassing puzzle requirements.
The experimental nature of WebMCP created several technical challenges. I worked through origin trial enrollment, browser flags, tool registration lifecycle issues, schema validation, agent client compatibility, and the differences between visual browser automation and true WebMCP tool usage.
The 3D world introduced additional challenges, including lighting dark environments without losing the science fiction atmosphere, making interactive objects distinguishable, improving interaction distance, supporting reliable keyboard controls, and presenting contextual actions without interrupting movement.
What I learned
I learned that providing tools is not enough to make an agent successful. Tool descriptions, observation quality, error responses, state visibility, and prerequisite feedback all have a major effect on agent performance.
I also learned that agents can become trapped in inefficient loops when an environment does not clearly distinguish between:
- Something that has not been discovered
- Something that is visible but out of range
- Something that requires another item
- Something that requires a derived password
- Something that cannot be interacted with
Designing good agent facing feedback is remarkably similar to designing good human user experience. The system must communicate what changed, why an action failed, and what kinds of reasoning remain possible without revealing the answer.
What's next
Agent Escape Room is designed as the foundation for a larger AI agent evaluation platform.
Future versions could include:
- Additional mysteries and environments
- Multiple difficulty levels
- Larger model comparison workflows
- Aggregate completion analytics
- Aggregate token and cost analytics
- Replayable agent action traces
- Standardized benchmark runs
- Custom escape room creation tools
My long term goal is to make agent evaluation feel less like running a static test suite and more like watching an intelligent system investigate a living world.
Log in or sign up for Devpost to join the conversation.