ExpertLens: SparseSight Studio

• 

Inspiration

Sparse-expert models introduce a new observability challenge.

A typical object-detection dashboard shows the final bounding boxes, but it does not explain what happened inside the model. It cannot tell you which experts were selected, whether the router is relying too heavily on a small number of experts, whether reduced precision changes routing decisions, or whether an apparent improvement still holds up against a stronger baseline.

Our research already produced many of the artifacts needed to answer these questions. What was missing was a clear, accessible product that a developer—or a hackathon judge—could understand within a few minutes.

ExpertLens was built to make that internal behavior visible.

What it does

Displays dense and sparse detection results side by side using bundled, redistribution-safe road scenes. Shows the top two selected experts for each scene, along with their routing weights. Visualizes routing-health metrics such as expert utilization, effective expert count, load balance, expert-pair coverage, top-k margin, and FP16/FP32 agreement. Compares latency, stored and active parameter counts, checkpoint size, and single-seed pilot AP across dense and sparse configurations. Sends a limited, public JSON evidence payload to GPT-5.6 and requests structured findings and one recommended next experiment. Calculates a checksum for the exact JSON payload so users can verify what evidence was analyzed. Provides a deterministic, evidence-aware report when an OpenAI key or network connection is unavailable.

How we built it

We built a clean, self-contained repository around a versioned evidence bundle instead of connecting the demo directly to our private research environment.

The interface is built with Streamlit. Detection overlays are rendered with Pillow, while the charts and visual components use custom HTML and CSS.

A lightweight Python layer handles the supporting logic. It validates the evidence bundle, calculates parameter and routing comparisons, prevents unsafe asset-path access, and creates a stable public JSON document for AI analysis.

For the live copilot, we use the OpenAI Responses API with GPT-5.6. The response is parsed into a Pydantic schema containing:

An executive summary Findings linked to specific evidence paths One practical next experiment Guardrails describing what the available evidence can and cannot support

Before the request is sent, the application calculates a SHA-256 checksum for the exact JSON payload. That checksum is displayed beside the generated analysis.

When credentials are missing—or when a network, API, or schema error occurs—the application automatically returns a deterministic version of the report instead of leaving the AI section unusable.

How we use OpenAI

GPT-5.6 is not given a vague prose summary and asked to make assumptions.

It receives only the structured public metrics already available in the interface. The prompt also includes a clear evidence policy:

Accuracy results must be described as pilot evidence. Latency claims must remain limited to the measured backend and protocol. Every finding must reference a supporting field in the JSON. The recommended experiment must include an option that can still be evaluated using cached data.

We use Structured Outputs so the analysis follows a consistent schema and can be rendered as a reliable part of the product interface.

The offline fallback follows the same evidence rules, which means the workbench remains useful even without an API key.

Challenges we ran into

The most difficult part of the project was not building the interface. It was maintaining evidence hygiene.

Our original research workspace contained private dataset references, experimental files, and checkpoints whose redistribution status had not yet been confirmed. We needed to preserve the strongest aggregate results without allowing private file paths, identifiers, or unsupported scientific claims to appear in the public repository.

We also had to clearly separate the demo scenes from the actual evaluation evidence. The scene replays are designed to explain the product and model behavior; they are not presented as proof of an accuracy improvement.

That distinction was important because a visually convincing demo should not be confused with a scientific result.

Accomplishments that we're proud of

Built a one-command, offline-first application with five connected workspaces. Presented real routing-health and compiled-systems evidence with clear scope and denominators. Created a typed GPT-5.6 copilot grounded in an auditable JSON payload. Added a useful deterministic fallback instead of disabling the AI experience when no key is available. Separated private research infrastructure from the public build. Included asset-provenance documentation, automated tests, and redistribution-safe inputs. Presented a negative or inconclusive accuracy result honestly instead of overstating the findings.

What we learned

Sparse-routing observability can be valuable even before a sparse model achieves better accuracy.

Metrics such as effective expert count, expert-pair coverage, load variation, top-k margin, and precision agreement help engineers determine whether the conditional-compute path is working as intended—and whether the model is healthy enough to justify further training and evaluation.

We also learned that a model can have a promising systems result while still having an inconclusive learning result.

In our case, the sparse architecture may offer an interesting efficiency and routing story without yet proving an accuracy advantage. ExpertLens is designed to make that difference clear rather than combining everything into a single success claim.

What's next for ExpertLens: SparseSight Studio

Our first priority is to freeze the public evidence bundle and verify it against the hashes of the original source artifacts.

After that, we plan to add a redistributable ONNX/CPU inference adapter once the checkpoint lineage has been reviewed and approved.

The next major research step is a matched, multi-seed comparison between learned routing and balanced-routing controls, using publication-quality implementation parity.

TensorRT support and a broader sweep across different expert counts are intentionally postponed until those validation steps are complete.

Built With

• gpt-5.6
• openai-responses-api
• pillow
• pydantic-structured-outputs
• pytest
• python
• streamlit