Domain Arena

Inspiration

I was curious how to get my website to show up in ChatGPT as a recommended source which led me down the path of Agent SEO. I think this is all still emerging and it was a fun exploration into what future useful tools could look like in this space.

What it does

Essentially its a simplistic version of experimenting with agents given this intent, which site are you most likely to browse to first. From this we can learn about how how agents will use and navigate the web vs humans and how to market to them better. E.G I learned that instead of more creative startup names like Porkbun more functional names are much more appealing to agents.

How we built it

The system has three layers:

1. Research layer — 16 experiments across 7+ model families (Llama, Mistral, Qwen, etc.) studying how agents discover, evaluate, and select tools. Key findings: description seduction is family-clustered, selection is contrast-driven not content-driven, serverless LLM inference is non-deterministic, position primacy dominates SERP choice.
2. Measurement layer — Pairwise AB/BA comparison with Wilson score intervals. Agents see two domains blind and choose one. We run 20+ trials per pair, randomize order to eliminate position bias, and compute confidence intervals. Cross-family replication ensures findings aren't model-specific.
3. Acquisition layer — The MCP server exposes 9 tools: search_domain, recommend_domain, compare_domains, prepare_registration, approve_domain, register_domain, configure_dns. A state machine with 11 states enforces strict transitions. Registration requires human approval via one-time token. Price drift guard invalidates approval if pricing changes. Hard budgets on purchase and renewal prices.

Challenges we ran into

Lots of experiments we tried gave no useful data.

Non-determinism in serverless inference. The same prompt produces different results across time windows on the same model. This means single-run measurements are unreliable. We had to build a multi-trial framework with statistical significance testing — one-shot scoring doesn't work.

Accomplishments that we're proud of

I think its a novel approach to a fairly new problem and one that will no doubt become more relvent over the next few years as agents become the primary users of the internet.

What we learned

Agents are better at relative judgment than absolute rating. When you ask "is this domain good?" you get noisy answers. When you ask "which of these two is better?" you get consistent, statistically powerful results. Pairwise comparison was the right methodological choice. Domain name style is noise when descriptions are clear. If the task description is specific ("A JSON repair tool"), agents infer function from context, not from the domain name. But when descriptions are ambiguous, name structure becomes a signal — and some models fall for "enterprise-sounding" names on broken tools.

What's next for Domain Arena

Execution-grounded selection (DA-T6) — measure whether agents can actually use the domain after acquisition, not just infer what it represents Bradley-Terry aggregation — richer pairwise models for ≥3 candidate comparisons Real-world outcome tracking — does acquiring an agent-tested domain actually improve agent performance? TLD effect isolation — measure whether .dev vs .com vs .io changes agent comprehension independently of the name itself

Built With

• cloudflare-workers-ai
• mcp-(model-context-protocol)
• mit
• name.com-api
• pairwise-ab/ba-comparison
• pytest
• python
• wilson-score-intervals