ThermalMesh Lab

Let AI agents understand, benchmark, and optimize heterogeneous inference infrastructure through WebMCP.

Comment

Inspiration

AI inference infrastructure is increasingly heterogeneous: workers can have very different capacities, yet simple strategies such as Round Robin still treat them as interchangeable. A slow worker can build a large queue while faster capacity remains available elsewhere.

At the same time, browser agents often interact with applications indirectly by inspecting interfaces, finding controls, and deciding what to click. For infrastructure software, however, operations like comparing routing policies or inspecting a bottleneck have much richer semantics than the buttons that trigger them.

ThermalMesh Lab grew from one question:

What if an AI agent could understand and operate inference infrastructure through domain-level tools instead of automating a dashboard by sight?

What it does

ThermalMesh Lab is a browser-based simulator for heterogeneous AI inference clusters.

Users can configure workers with different abstract capacities, generate deterministic workloads, and compare two routing strategies:

  • Round Robin — equal distribution without considering capacity or load.
  • Inference-Aware (ThermalMesh) — a simplified strategy using capacity, load, and estimated completion time.

The simulator exposes TTFT, queue latency, throughput, completed and unfinished requests, worker utilization, and queue depth.

The same application can also be operated by an AI agent through WebMCP Site Tools. An agent can configure the cluster and workload, run benchmarks, compare policies, inspect bottlenecks, and apply a routing strategy.

Humans and agents share the same application state, so every agent action is immediately reflected in the dashboard.

ThermalMesh Lab is a browser-based simulation, not a hardware benchmark.

How we built it

ThermalMesh Lab is built with React, TypeScript, and Vite and runs entirely in the browser.

The simulation engine generates a deterministic seeded request trace and gives both routing policies the same workload, making comparisons reproducible.

WebMCP is integrated through document.modelContext.registerTool(). The application exposes seven domain-level Site Tools:

  • get_cluster_state
  • configure_cluster
  • configure_workload
  • run_benchmark
  • compare_routing_policies
  • inspect_bottlenecks
  • apply_routing_policy

The WebMCP layer does not maintain separate agent state. Site Tools and the visible React interface call the same validated application/store operations.

We also implemented a state-dependent tool lifecycle. After a valid comparison, ThermalMesh Lab dynamically exposes an eighth tool:

apply_winning_configuration

If the cluster or workload changes, the comparison becomes stale and that tool is removed. In the deployed application this creates a live:

7 tools → comparison → 8 tools → scenario change → 7 tools

lifecycle driven entirely by application state.

The application requires no backend, database, authentication, or paid API and is deployed as a static site on Cloudflare Pages.

Challenges we ran into

The biggest challenge was making WebMCP a real application interface rather than a collection of tools that simply mimic button clicks.

Operations such as compare_routing_policies and inspect_bottlenecks needed to represent actual infrastructure semantics while keeping the human UI and agent perfectly synchronized.

Dynamic registration also introduced lifecycle problems. We had to prevent duplicate or stale registrations, ensure tools always operated on current state, and remove the winning-policy tool immediately when its comparison became invalid.

The simulator itself required careful handling as well. Both policies needed identical request traces, overloaded scenarios had to remain numerically safe, and unfinished requests could not appear as misleading zero-latency samples.

One of our first live agent tests produced an unexpected result: the agent realized its initial workload had completely saturated the cluster, reduced the workload, reran the comparison, and then selected a better routing policy. That showed us both the potential of semantic tools and the importance of precise tool descriptions when working with autonomous agents.

Accomplishments that we're proud of

We are especially proud that WebMCP is not an isolated demo layer. The human interface and the AI agent genuinely operate the same application.

In live testing, ChatGPT successfully discovered the Site Tools, configured the cluster and workload, compared routing strategies, analyzed the bottleneck, and applied the selected routing policy.

The state-dependent lifecycle also works end to end:

7 base tools → comparison → 8 tools → invalidation → 7 tools

We are also proud that the simulator does not artificially make Inference-Aware routing win. With homogeneous, lightly loaded workers, both strategies can produce effectively identical results.

Under heterogeneous load, differences emerge naturally. In one live test:

  • Round Robin completed 620 requests
  • Inference-Aware routing completed 741 requests
  • TTFT p95 fell from roughly 23.1 seconds to 1.7 seconds
  • Unfinished requests fell from 123 to 2

More importantly, the agent was able to explain why: Round Robin continued sending equal traffic to a much slower worker, creating a large queue, while Inference-Aware routing shifted work toward available capacity.

What we learned

The biggest lesson was that WebMCP becomes much more powerful when tools represent application semantics rather than UI actions.

A tool such as compare_routing_policies gives an agent far more reliable context than asking it to discover controls, run separate simulations, locate the results, and infer how the metrics relate.

We also learned that agent capabilities do not need to remain static. Exposing apply_winning_configuration only when a valid comparison exists gives the agent an interface that reflects what the application can meaningfully do at that moment.

The simulator taught us another important lesson: overloaded-system benchmarks can be misleading when unfinished work is ignored. A policy may appear to have better median latency simply because its slowest requests never complete inside the observation window. We therefore treat completion and unfinished work as first-class comparison signals rather than hiding them.

What's next for ThermalMesh Lab

The challenge version intentionally focuses on a small, reliable WebMCP experience.

Next, we would like to add additional well-defined routing baselines such as Weighted Round Robin and Least Queue, along with richer workload patterns, request classes, worker failures, and longer-running experiments.

ThermalMesh Lab could also evolve into a broader agent-native environment where agents design experiments, compare multiple strategies, explain tradeoffs, and iteratively search for better infrastructure configurations.

The broader idea we want to explore is simple:

Instead of teaching AI agents how to click through infrastructure software, make infrastructure software directly understandable to agents.

Built With

Share this project:

Updates

Submission history