WebCorp Autopilot

Autopilot patrols a real Latin American e-commerce network (318K orders), finds the lanes quietly losing money, and queues the fix. Honest statistics, human-approved actions, live on Qwen Cloud.

Comment

Inspiration

My family runs WebCorp, a 3PL logistics company (we run the delivery operation for online stores) across six countries in Latin America. Here is what makes this region different: a large share of e-commerce is paid cash on delivery (COD), meaning the customer pays the courier in cash at the door. If the courier fails to deliver, the package comes back and the money for that sale is simply never collected. "Effectiveness" is the percentage of orders a courier actually delivers in a zone, and when it collapses somewhere, revenue silently stops. I watched the same scene for years: failing delivery routes discovered ~3 days late, or never, because everything lived in Excel. On a single lane (Zacatecoluca × FORZA), $2,188 vanished in 90 days, and when we later measured the whole network, at least $36,440 per quarter was leaking across misassigned routes. I wanted a production-ready agent that watches the whole network every night, tells the truth about what it finds, and never acts without a human, running on Qwen Cloud, because Alibaba Cloud is expanding into Latin America and our region's e-commerce logistics is exactly the workload it should be winning.

What it does

Autopilot is a production-ready operational decision agent that automates an end-to-end business workflow on WebCorp's real order network: 318K orders, 1.6M tracking events. Detection of a failing lane went from ~3 days of manual Excel to seconds. Live demo, no login: http://8.219.56.30/panel/ (60-second test: click a question chip → watch the real tool-call trace travel the route map → open the Gate → approve with the two-step countdown → download the real PDF/Excel it generates).

  • Handles ambiguous inputs: ask "Why is FORZA doing so badly in Zacatecoluca?" and Qwen (function calling, multi-round loop) picks its own tools, self-corrects rejected arguments in-loop, and answers with evidence: Wilson LCB 23.4% (n=38), the worst statistically reliable pair in the network.
  • Leverages external tool APIs: 9 deterministic tools: a Wilson lower-confidence-bound estimator, alert engine, weekly series, zone/country aggregates, an exact MILP optimizer, a guarded web-search API, and one notification tool. The model never computes a number.
  • Smart human-in-the-loop verification checkpoints, enforced in code: any action with external effect is physically blocked: aprobado_por_humano can only be set by the approval panel (two-step confirm + countdown), not by the model, not by a prompt injection. On approval, deterministic code emits real artifacts (carrier-notice PDF, call-center recovery Excel, email drafts never auto-sent) in the operator's language (EN/ES/FR), and they stay on screen until the human confirms downloading them.
  • Autonomous patrol: a nightly cron scan asks standing questions; findings queue at the Gate for morning approval, deduplicated to one pending action per case.
  • The whole network at a glance: an interactive Latin-America traffic-light map (<50 red · 50–65 amber · ≥65 green) down to municipality level, where synthetic demo data declares itself row-by-row, never passed off as real.

How we built it

Qwen Cloud (Model Studio / DashScope): qwen-plus function calling drives the agent loop (agente.py, our proof of Alibaba Cloud deployment), with a trust layer in code: the external search only unlocks after internal evidence ran in the same loop; the notification tool sits behind the human gate with queue-level and send-level dedupe. Stack: Python + FastAPI + PuLP/CBC + SQLite, deployed on Alibaba Cloud ECS (Singapore) with systemd + nightly cron, and an append-only JSONL audit ledger where every event carries its origen_id: one click lights up the full chain of custody: question → tool → queue → approval → artifact.

The statistics refuse to lie, mechanistically. Every courier×zone pair ranks by the Wilson lower confidence bound (95%), computed from exactly two inputs per pair, successes $s$ and sample size $n$, with $\hat{p}=s/n$ and $z=1.96$:

$$\hat{p}^{\,LCB} = \frac{\hat{p} + \frac{z^2}{2n} - z\sqrt{\frac{\hat{p}(1-\hat{p})}{n} + \frac{z^2}{4n^2}}}{1 + \frac{z^2}{n}}$$

Sample size drags the estimate toward caution, so a lucky 100% (n=1) never outranks a proven 85% (n=53). Adopting the LCB changed which courier "wins" in 12 municipalities versus raw rates. The alert engine applies deterministic thresholds over these LCB values, the same ones the map shows (<50% critical · 50–65% watch), so an alert is a rule firing on a formula's output, never a model's opinion.

The optimizer, mechanistically. A generalized-assignment MILP solved exactly (PuLP + CBC):

$$\max \sum_{z}\sum_{c} v_z \cdot \hat{p}^{\,LCB}{c,z} \cdot x{c,z}$$

subject to: each zone served by exactly one courier ($\sum_c x_{c,z}=1$); no courier grows beyond $(1+\gamma)$ of its current volume ($\sum_z v_z x_{c,z} \le (1+\gamma)V_c$); $x_{c,z}\in{0,1}$; and only observed pairs with $n\ge30$ enter the model; we never extrapolate to unseen courier×zone combinations. Here $v_z$ is zone volume, $\hat{p}^{\,LCB}_{c,z}$ the Wilson 95% lower bound for courier $c$ in zone $z$, $V_c$ the courier's current volume, and $\gamma$ the capacity slack (25%). CBC certifies optimality: "exact" is a proof, not an adjective.

The headline number is the conservative one. The +921 deliveries/quarter (+6.82%) ≈ $36,440 USD recovered, with only 14 changes is projected with the Wilson LCB, the pessimistic bound. Re-evaluating the same optimal plan with raw rates yields +909: within ~1%, so the gain is not an artifact of the metric. And $\gamma$ barely matters. Real sensitivity runs: +10% → +893 · +25% → +921 · +50% → +924. The plan is driven by evidence quality, not by the capacity assumption.

Challenges we ran into

  • Qwen gotchas verified empirically: thinking mode is incompatible with tool calling (enable_thinking:false); search_info sources are only exposed by the native DashScope endpoint, not compatible-mode, so our search tool calls the native endpoint to cite real URLs.
  • We caught the model hallucinating: in testing, Qwen invented a plausible press citation without calling the search tool. We shipped the fix as a system-prompt rule plus a code guard: external context only exists if the tool returned it. Now, when the search finds nothing, the agent answers "no external cause found; the cause is internal until new evidence."
  • Honest optimization is more than solving the MILP: we had to declare the transferability assumption (courier performance holds under a ≤25% volume shift; congestion is not modeled), the ~90-day snapshot staleness (the plan re-solves with each estimator refresh), the feedback loop (reassigning by LCB starves the losing courier of new data in that zone; an explore/exploit split, bandit-style, is our documented mitigation), and the gaming risk (a measured carrier could mis-mark doubtful deliveries; the per-order ledger enables spot audits).
  • Privacy with real data: the production database is deliberately NOT deployed publicly; tools fall back to synthetic data that declares itself in every response, and client PII never passes through the LLM.

Accomplishments that we're proud of

Why we believe this project stands out, criterion by criterion:

  • Technical depth and engineering: every number is computed by a deterministic tool, never by the model; the assignment optimizer is an exact MILP with certified optimality (CBC), validated with a sensitivity analysis and a raw-rate cross-check; the human gate and the evidence-first search guard are enforced in code paths, not in prompts.
  • Innovation and AI creativity: to our knowledge, ranking courier assignment by the Wilson lower confidence bound and feeding that pessimistic bound into an exact optimizer is not how logistics dashboards work today; the LLM's role is inverted (orchestrator of tools, never decision engine), and the architecture is the UI itself: the route map the judge sees IS the system diagram, animated live.
  • Real-world value and impact: a real company with 318K orders runs on this; the measured detection improvement is ~3 days to seconds, and the optimizer's conservative projection recovers $36,440 per quarter with only 14 route changes.

  • Presentation and documentation: a no-login live demo with a 60-second test path, full math in the README, 8/8 token-free tests, and every claim in this page traceable to a formula, a code file, or the audit ledger.

  • Production-ready and in production: a real company runs on this: verified Alibaba Cloud deployment, live through the whole judging window, with a rules-only fallback if the LLM is unreachable.

  • $36,440/quarter found by an exact optimizer under the pessimistic bound, robust to the metric (±1% vs raw rates) and to the capacity assumption (±3% across $\gamma \in [10\%, 50\%]$).

  • A human gate that survives prompt injection because it lives in code, not in the prompt.

  • An agent that would rather say "I don't know" than invent a reason.

  • Fully trilingual (EN/ES/FR): UI, agent answers, and even the generated PDFs/Excels follow the operator's language.

  • 8/8 tests covering the loop, HITL, dedupe, fallback and artifact generation, without spending a single token.

What we learned

The hard part of an autopilot agent isn't the model; it's the honesty architecture around it: deterministic tools for every number, evidence-first guards for external claims, blocking checkpoints in code, declared assumptions and synthetic fallbacks, and an append-only trail that makes every decision addressable. We also learned to state our math mechanistically: an "exact optimizer" you can't verify is just an adjective, so the full formulation, the sensitivity table and the raw-rate cross-check are in the README.

What's next for WebCorp Autopilot

Deliberate non-goals today: no multi-agent negotiation, no demand forecasting, no auto-execution of reassignments (the human Gate is the product), no public production database. Next: explore/exploit volume splits to keep losing-courier estimates alive (the feedback-loop fix), Bayesian shrinkage for thin municipalities, webhook notifications to live ops channels, and multi-instance queues. And the bigger picture: Alibaba Cloud just opened its first Latin American region, and Latin American e-commerce logistics is one of its most underserved frontiers. Autopilot is a repeatable landing pattern for Qwen Cloud in the region: it ports to any logistics operator by swapping a single CSV. We built the bridge; now we want to carry traffic over it.

Built With

Share this project:

Updates