Tollgate

An edge visual inspection agent that decides when to call the cloud by cost rather than confidence, so privacy, offline resilience, and cloud orchestration all fall out of one decision.

Comment

Inspiration

Factory visual inspection systems keep re-solving the same three problems as if they were unrelated. They try to keep camera data private, stay running when the network drops, and decide when a small on-device model should defer to a larger cloud model. In almost every deployment these are bolted on as separate features, and the decision about when to escalate to the cloud is a hand-tuned confidence threshold that nobody can really justify. We wanted a single principle that produces all three behaviours as consequences, with an escalation rule derived from real operating costs instead of guessed.

What it does

Tollgate is an edge inspection agent that routes frames to the cloud by cost, not confidence. A confident part is decided on the device in milliseconds and never leaves the building. An uncertain part sends only a cropped region of interest to Qwen3-VL on Alibaba Cloud, which returns a structured verdict with the defect type, a confidence score, and a root cause. When the network drops, escalations defer to an on-device outbox and reconcile on reconnect, so the line never stalls.

The whole policy is one derived inequality. Given a local calibrated defect probability $p$, the cost of deciding locally is

$$\text{cost}{\text{local}}(p) = \min\big(C{FP}(1-p),\; C_{FN}\,p\big),$$

and escalating is worth it only when that cost, minus the cloud's residual error, exceeds the price of a cloud call. Solving each linear branch for the threshold $T = C_{\text{cloud}} + \varepsilon$ gives the escalation band

$$\text{band} = \left[\frac{T}{C_{FN}},\; 1 - \frac{T}{C_{FP}}\right].$$

Nothing here is hand-tuned. The band is computed from the operating costs and updates the moment they change, and privacy, offline support, and cloud orchestration all follow from this single decision.

How we built it

The edge runtime is Python 3.11. A thin orchestrator wires perception into the router, then the privacy filter, the cloud client, actuation, and a SQLite audit log. All of the policy lives in one router module, so everything else stays simple and swappable behind a common interface.

Perception is one ONNX classifier that is dispatched by input width, which keeps the live path in lockstep with training. The core backbone is DINOv2 ViT-S/14, with MobileNetV2 and a handcrafted feature set kept as ablation arms. A calibrated logistic head with temperature scaling turns embeddings into the probability $p$.

The cloud tier runs on Alibaba Cloud. The module cloud/qwen_reason.py calls Qwen3-VL through the DashScope OpenAI-compatible endpoint, validates a strict JSON schema on the server so the edge always gets well-formed output, and retries Alibaba rate limits with exponential backoff and jitter. The same logic is exposed both as an MCP tool named diagnose_defect and over HTTP at /healthz and /diagnose, served from a Docker container on an Alibaba Cloud SAS instance behind Caddy auto-TLS at https://tollgate.duckdns.org.

For resilience, a network controller probes the server per frame and sets a full, degraded, or offline mode. In-band escalations defer to a SQLite outbox when offline and back-fill the cloud verdict on reconnect. The project page is built in React and TypeScript, with interactive figures where you can drag the costs and watch the band recompute, run a frame through the pipeline and cut the network live, and a box that calls the deployed reasoner directly from the browser.

Challenges we ran into

An early unsupervised evaluation leaked training data into evaluation and reported perfect local recall everywhere. We rebuilt it to standardize scores by the distribution of good parts only, so clean parts pass locally and the numbers are trustworthy.

The offline path was originally two disconnected replays, so the network cut could not actually be demonstrated. We reworked it into a live state machine, where an escalation parks in the outbox mid-run and drains and reconciles when the connection returns, all on one continuous run.

Matching onnxruntime on GPU to the cluster's CUDA and cuDNN versions, and keeping train-time and live features on a single code path, took real effort to get right. The single cloud container also returned gateway errors under heavy concurrent evaluation, so we capped concurrency and added retry-with-backoff on server errors, which means the measured hybrid recall reflects the model rather than dropped calls.

Accomplishments that we're proud of

The escalation rule is derived rather than tuned, and the data backs it up. Across 23 categories of MVTec AD and its harder 2024 successor MVTec AD 2, the escalation rate is strongly anti-correlated with local recall, with $r$ around $-0.9$. The router spends the cloud budget exactly where the local model is weak and stays quiet where it is confident.

The reasoning tier is genuinely deployed, not mocked. Qwen3-VL runs on Alibaba Cloud behind HTTPS, and the project site calls it live in the browser with real verdicts. We measured it on real Qwen3-VL calls at full accuracy on our samples, and the whole hybrid recall figure uses live cloud verdicts, not simulated ones.

The router is modality-agnostic. The same cost decision works on 2D surfaces, logical anomalies in MVTec LOCO, and 3D point clouds in MVTec 3D-AD, with no change to the orchestration, the privacy filter, or the outbox.

What we learned

Deriving the escalation rule from costs instead of a confidence cutoff is what makes the system honest. The router escalates in proportion to how weak the local model is, which is exactly the behaviour you want and exactly what a fixed threshold cannot give you.

We also learned the value of reporting failures. On the hardest AD 2 categories the cloud model struggles too, so hybrid recall tracks local recall. We show this rather than hide it, because the routing decision is still correct and a stronger cloud model would lift the ceiling without touching the router. Keeping the local model, the modality, and the cloud model swappable behind one inequality turned out to be the real engineering win.

What's next for Tollgate

The immediate next step is to make DINOv2 the local backbone at runtime as well as in evaluation, so the live pipeline matches the benchmark exactly, with the trained head committed to the repository.

Beyond that, the router is ready for productization. Because the escalation band is derived from costs, an operator can retune $C_{FN}$, $C_{FP}$, and the cloud cost for a new part family or a shift change without retraining anything, and the band updates automatically. We want to add a second-opinion tier that routes the very hardest cases to a stronger cloud model, per-part cost profiles, and a small operator dashboard over the SQLite audit log so the zero-PII claim stays measurable in production. The whole thing is designed to be adopted, either as an open-source edge agent or as the routing core inside an existing inspection stack.

Built With

Share this project:

Updates