Aegis

Dashboard
Dashboard

Inspiration

Traffic congestion is a strange problem: the low-level control is a solved, millisecond-scale engineering discipline, but the judgment layer above is not. A stalled truck, a school-zone surge, and a concert letting out all look like "long queues" to a signal controller. They need different responses, and today that judgment lives in a human control room.

Our first thought was obvious: traffic is jammed, Qwen is smart, let Qwen pick the green light. But, that should not be implemented, a cloud LLM should never be in the hot loop of a safety-critical system that must survive bad network days. That rejection became the whole design: the edge is authoritative. Qwen proposes. Humans approve. The safety validator gets the final word.

What it does

Aegis is a traffic-signal EdgeAgent. A local edge runtime runs the intersection on a perceive → decide → validate → actuate loop. Qwen, running on Qwen Cloud, wakes up at event cadence: it reasons over incidents, SOPs, weather, calendar events, and telemetry (via RAG and MCP tools), then emits a schema-valid policy proposal. A human operator approves or rejects it in a dashboard. Approved policies are still clamped by a hard safety validator before they touch the lights. Cut the cloud link mid-demo, and the intersection keeps running on the last safe policy, then a cached fallback plan. Nothing leaves the edge, a privacy filter is enforced, so only relevant telemetry are passed to the cloud.

How we built it

Edge runtime + safety validator (Python). The validator is the only component that emits signal commands. It enforces min/max green, yellow/all-red clearance, and anti-starvation, and it clamps every policy, whether authored by Qwen or approved by a human.
Qwen operator (QwenCloud). A two-stage reason → emit pattern: free-form reasoning with tools first, then a constrained call that emits PolicyProposal JSON validated against a schema, with one repair attempt and a safe fallback. We measured model tiers instead of guessing: qwen3.6-flash matched qwen3.7-plus on proposal quality at roughly half the latency, so flash is the default and plus is the escalation model.
Escalation ladder. Deterministic triggers (zero LLM cost) → Qwen-VL confirms a cropped frame and grades severity → only high severity auto-escalates to the full proposal flow. The LLM gets called when something is worth reasoning about, not every tick.
Perception. An HDI-finetuned YOLO11 detector (exported to ONNX) counts vehicles in calibrated approach zones: 75.9% exact-bin and 94.6% busiest-approach agreement against across five clips. In the demo, SUMO carries the closed control loop; the CV path is a measured showcase that shares the same TrafficState contract and feeds Qwen-VL incident intake live.
Learned controller. We evolved a tiny NumPy MLP with a genetic algorithm instead of shipping our original DQN. Fitness is throughput-dominated, $f = \text{throughput} - 2 \cdot \text{in_network} - 0.5 \cdot \text{timeloss}$, evaluated inside the exact validated deployment loop. In a 2-hour live A/B it beat our actuated baseline: +21% throughput (1267 vs 1044 vehicles) and 71% lower mean wait (158s vs 544s).
Hosted backend. FastAPI + WebSocket backend on Alibaba Serverless App Engine; a full operator ask (RAG retrieval + five MCP tool calls + Qwen proposal) completes in seconds hosted.

Challenges we ran into

Every hosted Qwen call silently black-holed. After deploying to SAE, DashScope requests just hung, while in-process RAG kept working, which made it look like our code. The VPC had no internet egress. One NAT gateway, EIP, and SNAT entry later, the pipeline came alive. Lesson: check egress before debugging your client.
Rich reasoning and strict structured output fought each other in a single call, so we split the operator into reason → emit. The workaround became the architecture.

What we learned

The most valuable thing an LLM adds to a control system is not control, it's supervision: interpreting a situation, citing the right SOP, proposing a bounded policy, and explaining why. Everything hard about this project was keeping that boundary honest: in the safety validator, in the fitness function, in the demo script, and in our own claims.