InfraPilot WebMCP

💡 Inspiration

As large language models scale and multi-model inference pipelines (vLLM, SGLang, DeepSeek, Qwen) become the backbone of modern AI infrastructure, operating and orchestrating these distributed clusters has become increasingly complex.

Traditional DevOps observability relies heavily on human-centric graphical user interfaces—complex dashboards, nested SVG node graphs, and dropdown menus. When autonomous AI agents are tasked with managing or triaging cluster incidents via raw DOM scraping or vision models, they frequently suffer from hallucinations, missed metrics, or misclicks.

InfraPilot was born to bridge this fundamental gap using OpenAI's emerging WebMCP standard (document.modelContext.registerTool). Instead of forcing models to guess through UI pixels, InfraPilot transforms the web control plane into an agent-native environment where models directly discover and execute structured cluster management tools.

🚀 What It Does

InfraPilot provides a real-time, agent-native control plane for distributed LLM inference clusters. It registers four core atomic tools directly on the browser's document.modelContext:

1. get_cluster_topology: Exposes the complete topology of active inference engines, gateways, prefix KV caches, and vector databases in clean, structured JSON.
2. scale_service_replicas: Allows an autonomous agent or human operator to dynamically scale inference pods (e.g., DeepSeek-R1 or Qwen2.5) during sudden traffic surges.
3. optimize_gpu_memory: Triggers immediate eviction of stale prefix KV caches and reclaims fragmented GPU VRAM across nodes.
4. run_inference_benchmark: Automatically executes concurrent throughput stress tests to verify cluster latency and throughput post-optimization.

Every single tool execution is transparently recorded in a real-time Execution Audit Log, giving operators full observability with exact millisecond duration, input parameters, and structured outputs.

🛠️ How We Built It

• Frontend Architecture: Built with React 19, TypeScript, and Vite with a responsive Linear-inspired design.
• WebMCP Integration: Native registration against browser-level document.modelContext.registerTool with a development fallback bridge ensuring seamless evaluation across standard browsers, ChatGPT In-App Browser, and WebMCP-enabled Chrome.
• Hosting & Edge Delivery: Deployed globally on Cloudflare Pages for sub-millisecond edge response times.
• Styling: Styled using Tailwind CSS v4 with zero-compromise accessibility and dark-mode aesthetics.

🧠 Challenges We Ran Into

• State Synchronization & Zero-Latency Feedback: Ensuring that agent-driven WebMCP tool executions seamlessly update the React visual topology and the streaming execution audit log without UI lag or race conditions.
• Standard Protocol Compliance: Carefully designing the JSON Schema for tool inputs/outputs to guarantee maximum discovery reliability across both ChatGPT's in-app browser and Chrome's WebMCP experimental runtime.

🏆 Accomplishments We're Proud Of

• True Agent-Native UX: The cluster can be fully managed through natural language prompts via ChatGPT in-app browser (e.g., "DeepSeek latency is high, scale it to 4 replicas and free stale KV memory").
• Sub-Second Recovery Loop: Demonstrating how WebMCP enables autonomous closed-loop cluster healing (triage → scale → optimize → benchmark) in under 2 seconds without a human touching a mouse.

📚 What We Learned

WebMCP represents a paradigm shift in web development. Web applications are no longer just human-to-computer interfaces; they are becoming shared collaborative workspaces for both humans and AI agents. Building with WebMCP taught us the immense power of declaring structured API capabilities directly at the document level.

🔮 What's Next for InfraPilot

• Adding dynamic multi-cluster federation support (AWS, CoreWeave, Lambda Labs).
• Integrating autonomous anomaly-detection background triggers that proactively invoke WebMCP optimization tools before SLA breaches occur.

Built With

• agentic-ai
• cloudflare-pages
• deepseek
• openai
• react
• tailwind-css
• typescript
• vite
• vllm
• webmcp