MoE Autopilot Studio

A Windows lab that turns measured MoE routing evidence into a reproducible ENABLE, DISABLE, or MEASURE decision.

Comment

Inspiration

Local mixture-of-experts inference has a workload-dependent placement problem. Keeping hot experts resident can improve generation, regress prompt processing, exceed the WDDM VRAM margin, or look faster only because two incompatible builds were compared. Existing benchmark files do not answer the operational question: should this workload enable the split, with which measured configuration, and what experiment should run next?

What it does

MoE Autopilot Studio loads sanitized measurements, computes canonical expert coverage, checks immutable protocol fingerprints and hardware budgets, and produces a deterministic ENABLE, DISABLE, or MEASURE verdict. It exposes coverage, decode, prefill, total latency, break-even, RAM, and VRAM in one Windows interface. A safe local runner launches only known llama.cpp tools as argv arrays, records the exact environment and outputs, and persists results locally.

The offline fixture path works in under two minutes without Python, a model, a GPU, an account, or network access. The Windows release adds custom imports and measured A/B runs.

How we built it

The typed engine, protocol guard, evidence model, Windows Studio, safe runner, imports, exports, tests, CI, packaging, and hosted fixture report were built during Build Week. The optional explanation layer uses Codex App Server over stdio and ChatGPT OAuth. Each analysis starts an ephemeral GPT-5.6 Sol thread in an empty read-only directory with approvals disabled. The model receives only user intent and a bounded deterministic report. It may select only an experiment ID emitted by the engine, cannot change the verdict, and is rejected if it introduces an unsupported number.

Before Build Week, a separate research repository contained the load-time hot/cold llama.cpp fork, profiler, hot-list format, coverage convention, and workstation measurements. This submission does not claim a new cache algorithm, dynamic caching, training, or DeltaMoE.

Challenges

The core challenge was making a useful advisor without allowing the model to rewrite measured truth or execute arbitrary tools. The Studio also had to detect incompatible benchmark arms, respect Windows/WDDM memory margins, preserve exact provenance, and remain useful entirely offline.

Accomplishments

  • Deterministic workload-aware ENABLE, DISABLE, or MEASURE decisions
  • Protocol-fingerprint guards that reject invalid historical comparisons
  • A safe allowlisted runner with local evidence history
  • GPT-5.6 explanations constrained to engine-emitted facts and experiment IDs
  • A credential-free judge path plus a packaged Windows release and hosted report
  • Public evidence showing 68.56% coverage, +22.22% decode, -9.90% prefill, and a derived 19.18 prompt/output break-even

What we learned

A routing optimization cannot be judged by decode speed alone. Prompt shape, total latency, memory margin, build identity, repetition count, and replayable evidence all matter. The best role for GPT-5.6 here is explaining and prioritizing deterministic experiments, not replacing the measurement engine.

What's next

Expand the dated provider and hardware matrix, add more measured workloads and repetition statistics, strengthen migration/backup coverage, and keep new optimization research separate from the evidence-driven control surface.

Built With

Share this project:

Updates