Agent Arcade

An inspectable playground where AI agents solve logic and visual puzzles.

Comment

Agent Arcade

Agent Arcade is an interactive web playground for observing AI agents solve logic and visual puzzles. It makes an agent's work inspectable: a user selects a challenge, brings their own model key, starts an agent, and watches a live journal of board observations, plans, explorer branches, helper scripts, validated actions, cost estimates, and the final result.

The problem

Computer-use and reasoning agents are hard to evaluate from a final answer alone. A model can reach the right outcome for the wrong reason, silently rely on an answer key, or fail without revealing where its strategy broke down. Existing evaluation suites are often aimed at researchers rather than people who want to see an agent actually work.

What Agent Arcade does

Agent Arcade turns small but meaningful puzzles into controlled micro-benchmarks for planning, spatial reasoning, visual matching, and computer use.

  • Logic worlds: Sokoban, maze, and Klotski challenges with rule-validated actions.
  • Visual worlds: real-image Jigsaw boards that require visual fragment matching, plus cursor-controlled Jigsaw and Klotski variants.
  • Inspectable agent loop: models can observe a board, publish concise public checkpoints, make short action batches, spawn focused explorer branches, and write/run small Python helpers in an isolated algorithm sandbox.
  • Visible execution: actions animate on the live board; the journal distinguishes public observations, plans, helper work, explorer reports, placement decisions, and results without exposing hidden chain-of-thought.
  • Bring your own key: provider keys stay in the browser session and are not persisted by the app. Anthropic currently powers live puzzle agents; the OpenAI connection is included for BYOK setup and model validation.
  • Deployment-ready: the app is live on Render, so a judge can open the URL, add their own key, choose a puzzle, and run an agent.

How to use it

  1. Open the hosted app and choose a puzzle family and level.
  2. Open API keys and add a personal provider key for the session.
  3. Open Agents, choose a model and agent mode, and run it.
  4. Follow the live journal and board animation. Explorer branches appear in the agent panel; View trace shows the complete public run record.

Technical implementation

The project is a lightweight Node.js web application with a vanilla HTML/CSS/JavaScript interface and a server-side event stream for long-running agent calls. Every puzzle implements a constrained environment with an observation, action validator, solved-state check, and renderer. Logic puzzles use explicit validated actions; visual environments render a virtual screen and, for computer-control variants, validate literal cursor drags. Jigsaw uses real image fragments rather than an exposed answer-key mapping.

The agent runtime is designed as a reusable loop: observe → publish/plan → optionally explore or write a sandboxed helper → act → render → repeat. It records token use and estimated API cost alongside the human-readable trace.

Multi-agent workspaces and parallel exploration

Agent Arcade is built around one authoritative puzzle workspace and multiple isolated explorer workspaces. The lead agent can delegate distinct questions to explorer agents—for example, testing a left-crate route versus a right-crate route in Sokoban, checking a potential deadlock, planning a route, or examining an ambiguous visual region in a Jigsaw.

Each explorer receives a snapshot of the current puzzle state and its own private workspace, trace, and, for visual tasks, virtual cursor. It can investigate an approach without changing the real board or interfering with another explorer. Explorer branches can be scheduled in parallel where the model provider and API limits permit, then return concise reports and candidate plans to the main agent. The main agent decides whether to use a report; any proposed move is still revalidated against the current authoritative board before it visibly changes the puzzle. This provides genuine multi-agent exploration without letting competing experiments corrupt the shared run.

The interface makes those branches inspectable: the Agents panel exposes explorer status and the public journal records branch start, objective, report, helper work, and the main agent's eventual action. The live runner deliberately bounds explorer count and nested work to make collaboration observable and control API cost.

Built with Codex and GPT-5.6

I used Codex with GPT-5.6 throughout the project to turn the original product sketch into the working application: shaping the environment interface, implementing the puzzles and renderer, building the agent event loop and sandbox, iterating on the visual design, diagnosing live-model failures, testing the deployed experience, and preparing the submission materials. The result is intentionally not just a static demo: it is a live, inspectable agent arena that other people can use with their own keys.

Most implementation happened in one Codex session using GPT-5.6 Terra at Extra High reasoning. Codex also helped produce the project reports, demo-video edit, captions, and AI-generated narration.

Contest credit usage

I used approximately 1,900 of the 2,500 Codex credits provided through the contest to build Agent Arcade end-to-end. At the contest conversion of 2,500 credits = $100, this is equivalent to approximately $76 of the $100 allocation. That investment covered the iterative product design, implementation, visual and interaction refinements, agent/runtime debugging, browser testing, deployment, documentation, and demo-video production.

Links

What's next

The same observation/action harness is intended to grow beyond the built-in arcade. A future browser-extension adapter can provide screenshots, DOM/accessibility state, clicks, typing, and drag actions for real websites while preserving the same visible trace and benchmark experience.

Built With

Share this project:

Updates