Playable Arena

The benchmark already ran. Your product question didn't. WebMCP lets the agent continue from the published evidence.
The benchmark has 22 detailed checks. The agent highlights the seven that matter for building a replayable strategy game.
The agent opens four shortlisted games and completes the same level in each one using WebMCP.
The games are shown without names or scores. The person plays the shortlist and chooses by feel and taste.
The result becomes a concrete harness plan for the next run: what to preserve, which checks to add, and what to review with a human.

The benchmark already ran. Your product question didn't.

Playable Arena is a benchmark you can keep evaluating after the leaderboard is published.

Benchmarks such as SWE-bench and Terminal-Bench answer a valuable but fixed question: under the same tasks, checks, and agent setup, which system performed best? A developer choosing an agent for a real product asks a different question: which failure modes matter for my workflow, which output qualities feel right, and what should my agent test next?

Arena preserves both kinds of evidence. Fixed checks measure whether each result meets the requirements. Playable outputs preserve qualities a score cannot settle: clarity, responsiveness, taste, and fit for a particular audience.

WebMCP lets the AI agent in your browser continue that evaluation inside the same Arena page. It reads detailed evidence, updates the comparison you can see, and tests the same scenario across shortlisted outputs. You then compare them anonymously and make the subjective choice.

Arena turns the fixed benchmark, live verification, and your decision into acceptance checks for the next agent run. The published score never changes.

In the recorded demo, the agent reduces 22 published checks to seven relevant criteria, then completes the same six-move, three-push level in all four shortlisted builds. The result is not another leaderboard. It is a concrete plan for what the next run should preserve, test, and improve.

The current release proves this pattern with five standardized browser-game tasks and 70 playable benchmark outputs. Today those outputs are produced by coding-agent configurations. The recorded demo uses Codex. Arena supports both WebMCP surfaces: document.modelContext in ChatGPT/Codex in-app browsers and navigator.modelContext in Chrome.

The shared evaluation loop

Read the evidence. The agent uses WebMCP to select the published checks that matter for the person's goal and updates the comparison they can see.
Test the playable outputs. The agent opens the shortlist and completes the same level in each game. Agent Play verifies facts the table alone cannot: whether state is readable, actions are accepted, out-of-date actions are rejected, and a complete run works.
Return the decision to the person. Arena presents the builds anonymously. The person plays them and chooses in the UI; only then are identities and published scores revealed.
Improve the next run. Arena combines the fixed benchmark, live verification, and human choice into review stages and acceptance checks for the next agent run.

Arena does not claim that one model or agent setup is universally best, and it never changes the published benchmark score.

Why WebMCP matters

WebMCP can automate a workflow end to end. Arena uses it to automate the parts of evaluation that should be automated: reading evidence, replaying scenarios, and gathering results, while keeping subjective judgment with the person.

The agent and person work on the same live page. Tool calls update the visible filters, selected builds, game state, and anonymous review. The available tools change with context: benchmark tools appear on the benchmark, comparison tools on a task, game actions only for a compatible active build, and the result tool only after the person chooses. Arena never exposes a tool that lets the agent make the subjective choice or reveal identities early.

Without WebMCP, the person would have to read a large result table, open builds one by one, manually repeat scenarios, copy observations into another chat, and reconstruct the comparison. With Arena, the agent handles breadth, interpretation, and repeatable checks while the person contributes the judgment automation cannot supply.

The public benchmark stays fixed. What changes is how a developer can use it: what to preserve, what to borrow, which reviews to run, and what must pass next.

How it works

Arena registers tools for the current page with modelContext.registerTool(...). When the person moves to another page, the old tools are removed and the relevant ones appear.
Generated games run in isolated frames. A small arena.game.v1 agreement lets Arena expose only the actions owned by the current task.
Each action carries a revision number, so an out-of-date action is rejected without changing the game.
Anonymous review provides no tool that can choose or reveal on the person's behalf.
Production runs on Cloudflare Workers. D1 stores anonymous review choices, while a separate Artifact Worker serves the playable builds from another origin.

What existed before the challenge

Before August 25, Arena was a private playable benchmark. It could give coding-agent configurations the same brief, score their results with fixed checks, keep the games playable, and let a person compare two results anonymously.

It had no WebMCP tools, no way for an agent to inspect the detailed evidence or play the builds, and no path from a person's choice to checks for the next run.

What I built during the challenge

I built the missing end-to-end flow:

Added page-specific WebMCP tools so the agent can read evidence and update the comparison visible to the person
Added Agent Play so it can read live game state and repeat the same scenario across very different builds
Rejected out-of-date actions without changing the game
Connected agent-selected shortlists to a human-only anonymous choice and reveal
Turned the fixed results, live verification, and human choice into review stages and acceptance checks for the next run
Rebuilt the public Benchmark, task, comparison, evidence, navigation, and mobile experiences
Separated the public product from private briefs, raw runs, credentials, and production history
Added automated leak checks and deployed the UI, playable builds, and anonymous choice storage on Cloudflare

I also expanded the working testbed from 3 tasks, 39 builds, and 13 agent configurations to 5 tasks, 70 builds, and 15 configurations.

Adding LUMEN YARD and SHOAL meant more than adding two game cards. Each task needed one shared brief, fair checks for every build, independent runs across multiple agent configurations, playable published results, and a shared set of named actions so an agent could repeat the same scenario across implementations that shared no code. The new tasks expose differences in strategy, recovery, board readability, and action feedback that a top-line score alone cannot explain.

I also added Antigravity + Gemini 3.8 Flash and fx + GPT-5.6 Sol configurations.

HACKATHON.md records the plain-language before/after boundary and dated commit evidence.

Try it

Open Playable Arena in a WebMCP-capable in-app browser. No Arena account or MCP server URL is required.

Ask your agent:

Using this benchmark, what should I pay extra attention to when using Codex to build a replayable strategy game? Read the detailed checks, play the relevant candidates, and prepare an anonymous review. Do not choose for me.

When the anonymous review opens, play the candidates and make the choice in Arena. The agent cannot make that choice for you.