Second Mayor: build a city with your agent

• 

Inspiration

I wanted to play a city-builder with an AI agent. I love this type of game. Since my childhood I've been a huge fan of SimCity, and more recently Cities: Skylines.

I didn't want to watch a demo of an agent filling in a city. I wanted real work between me and an agent. The hackathon was short, so I needed a game I could build on instead of starting from zero.

So I went looking for a city builder I could open up, and found IsoCity: open source, TypeScript, MIT licensed, and built on plain canvas with no game engine underneath. That meant I could reach into the game state directly and wire WebMCP into it.

What it does

I've added a new mode to IsoCity. It's called Co-Mayor.

It creates a portal for your little agent! With this portal, you have two modes: Advisor and Co-Mayor.

The first mode, Advisor, lets you ask your agent questions about your city. He can advise you on anything. If you're wondering why your city seems unsafe, ask him! Why are you losing money? Ask him. If there are traffic problems, ask him.

The second mode is Co-Mayor, and it's a step up from the first one. This time your agent can make changes to your city! For example, when you start a game, just ask him to draw roads optimised for revenue at the top of the map. He will gladly help you do this.

WebMCP is the only way this works. The whole game is drawn on a canvas, so an agent that clicks buttons and reads the page sees nothing at all. Tools are not a shortcut here, they are the only way in. And because every build goes through a plan I approve, the agent can do real work in my city without me losing control of it.

How I built it

The base game is IsoCity by amilich, MIT licensed. Next.js, TypeScript, HTML5 Canvas.

I played it first. I wanted to know what a mayor actually does in this game, because that list was going to become my list of tools. You zone land, connect it with roads, keep power and water running to it, set a tax rate, fund departments, and then watch what breaks.

My goal was to expose as much as possible through WebMCP. Not only the actions, but the situation: the terrain, the budget, so he can understand what's right and what's wrong. Everything YOU already read to make a judgment on your city should be exposed to WebMCP. In total there are 22 tools exposed.

Then I looked at what the game already gave me. IsoCity keeps its state and its actions in one React context. Place a tile, bulldoze, set the tax rate, fund a department, change the speed, make the map bigger. The buttons in the UI call these functions. So I did not have to build an action layer. Every button was already a tool the agent could use, if I gave it a way to ask.

Then I made the list of tools before I wrote any. Two columns. What I want the agent to see, and what I want it to change. I did the seeing column first, and that turned out to be most of the work.

For seeing: the land around a point, the city stats, what is going wrong right now, and its own status. For changing: nothing happens directly. The agent proposes, and the plan waits. I did not want an agent editing my city behind my back.

Then I wrote the code. src/context/AgentContext.tsx handles the plan: propose, ghost, confirm, undo. src/hooks/useWebMCPTools.ts registers everything with document.modelContext.registerTool. The tools read the game through a live ref, not React state, so the agent always sees the city as it is right now.

Reading was the harder half. The map is a canvas, so there is nothing in the page for an agent to look at. It only knows what I hand it. get_problems is where most of that went: buildings with no power or water, traffic, fires, abandoned tiles, empty zoned land, and notes on what the city wants. Without it the agent builds in the wrong place and feels sure about it.

Challenges I ran into

I ran into multiple challenges. The most obvious one is that React is not a game engine. The game state changes every tick, so the agent needs to read live information from the game. I fixed it by letting the tools read through a live ref rather than the rendered state.

Another challenge was testing locally. I kept thinking everything was broken. The issue was that ChatGPT couldn't reach localhost, so I had to use the ChatGPT in-app browser.

An AI can also do many random things, so I had to guide my agent and point it in the right direction. I had to rename some tools so it wouldn't get confused. It would send plans far bigger than the game could take, then have no idea why they came back smaller. So the limit is published now: get_tool_catalog and get_agent_status both report 'maxPlanTiles: 200', and propose_placements answers with truncated and 'droppedForCap' so the agent can see exactly what got cut.

Applying a plan was where everything broke. Plans sometimes applied themselves before I approved them. Plans sometimes did things that should not have been legal in the first place. Part of it was WebMCP delivering the same call twice, so confirm_plan now keeps applied plans in a cache and returns 'alreadyApplied' instead of building the same thing again.

The agent did not know what it could afford. It once decided on a plan when I had no money left. It destroyed my city more than once. That is not the agent being stupid, that is me not giving it the game's constraints. A lot of the work turned into passing the current context down: what state the game is in, what the rules are right now, what is affordable right now.

That is also why undo had to be good. Applying a plan compares the grid before and after, so undo puts back what actually changed. It only touches a tile if that tile still looks the way the agent left it. If you built a school on top of the agent's road, undo leaves your school alone.

Accomplishments that I'm proud of

I'm proud that I can play a city-builder with an AI. This hackathon let me dig deep into WebMCP and into the game itself. I learned a lot, and the result is playable and fun to play with an agent.

What I learned

Tool descriptions are the most important thing in WebMCP and MCP. Every time my agent did something weird, the fix was mostly in the tool descriptions.

The number of tools matters just as much. When I started, I tried to add as many tools as I could. The more I worked on this project, the more tools I deleted, to keep things small and working.

What's next for Co-Mayor

Today the agent starts fresh every time. I want him to remember the city between sessions: what we built together, what we already tried, what you like. A co-mayor who forgets your city every morning is not much of a partner. So a real memory system for the agent.

Next I want the agent to disagree with me. Right now he does what I ask. It would be more interesting if he could say a plan is a bad idea and explain why, and I have to convince him or change my mind.

Built With

• codex
• next.js
• openai
• vercel