PreMortem

Inspiration

Every autonomous agent eventually faces an action it cannot undo. In accounts payable that action is paying an invoice: once the wire leaves, a swapped bank account or a tampered invoice is real money gone. And it is not hypothetical — the FBI's IC3 counted $2.77 billion in reported business-email-compromise losses in 2024 alone (21,442 complaints), and titles its own decade-view advisory "Business Email Compromise: The $55 Billion Scam". The canonical BEC play is exactly this failure: a payment instruction whose bank account was quietly swapped. The standard agent pattern decides by asking the model "is this plan good?" — a confirmation-seeking question a fluent model answers yes. We wanted the opposite reflex: an agent that, right before the irreversible step, spends its compute trying to falsify its own plan.

What it does

PreMortem gates irreversible actions behind a pre-mortem:

1. Two regimes. Reversible actions (draft, preview, read, simulate) execute immediately — no ceremony. Only irreversible/unknown actions (paying money) trigger the pre-mortem.
2. Enumerate how it could be wrong. It assembles failure modes from three sources: a fixed registry of 11 known AP catastrophes, an append-only memory of past real failures, and extra modes proposed by qwen-max.
3. Try to confirm them. For each mode it runs a read-only probe — payment history, vendor registration, account diffs, and crucially a qwen-vl-max reading of the invoice image cross-checked against the payment instruction.
4. Decide by falsification. If it confirms a danger at or above the block threshold, it stops and escalates. If a danger it cannot check is catastrophic or remembered from a past real failure, it escalates rather than guess an unverifiable payment; lighter unprobeable risks are surfaced as a non-blocking advisory. It pays only when it attacked its plan and could not break it.
5. Learn. Every block, human override, and post-hoc miss is appended to failure memory, so the next pre-mortem on a similar payment is seeded by what actually went wrong before.

The one-line thesis: PreMortem executes an irreversible action only when it has tried to prove the action wrong and failed — and it remembers every failure so its next pre-mortem improves itself.

Where the verdict flips — five scenarios, one engine

Most invoice autopilots automate the happy path and put a blanket human sign-off at the end of every run. PreMortem spends its intelligence in the opposite place: it attacks its own plan first, and interrupts a human only when it confirms a danger or cannot rule one out. The difference is visible in five bundled scenarios (deterministic on the offline mock — reproduces byte-for-byte):

The two ❗ rows are the thesis: a confirmation-seeking agent pays them; PreMortem proves its own plan wrong first. tampered_img is the check a text-only competitor structurally cannot build.

How we built it — and where Qwen is irreplaceable

The falsification engine stands on three legs: qwen-max enumeration × qwen-vl-max image grounding × deterministic non-LLM probes. The vision leg is the one a text-only competitor cannot build: in the tampered_img scenario the structured payment looks perfectly clean, but the invoice image shows a different bank account. qwen-vl-max (MultiModalConversation, vl_high_resolution_images for dense invoices) reads the document and the image-consistency probe catches the mismatch the structured data hides.

Qwen Cloud exposes no logprobs, which closes the naive "ask the model how confident it is" path — and that constraint is the design. Confidence comes from evidence: self-consistency sampling (N=5) on the enumeration leg plus the read-only probes, with the verdict anchored on the non-LLM probe leg wherever one exists (the two correlated LLM legs are demoted to a pre-filter). Crucially the sampling is wired into the verdict, not just measured: when the model cannot agree with itself (modal agreement < 0.6) about an irreversible payment's risks, the engine injects a catastrophic, unprobeable llm_enumeration_unstable mode that escalates to a human through the very same falsification rule as any danger it cannot check — so an unstable risk-read can never be silently cleared into a payment. The probe bank (all eleven read-only checks) is also exposed as MCP tools, so any Qwen-Agent can call the falsification checks to ground its own reasoning.

• Reasoning: qwen-max via the DashScope OpenAI-compatible endpoint.
• Perception: qwen-vl-max via DashScope MultiModalConversation.
• Model compute on Alibaba Cloud: all model reasoning (qwen-max) and invoice perception (qwen-vl-max) run on Qwen Cloud / DashScope — an Alibaba Cloud service, called from dashscope_adapter.py. The FastAPI surface (premortem.api:app) is a thin local API in front of that Alibaba-Cloud compute — every irreversible decision is reasoned and the invoice perceived on Alibaba Cloud.
• Provider abstraction: a QwenClient ABC with a deterministic mock (creds-free default) and the real DashScope adapter — the engine code is identical in both, so every tested verdict is the verdict production runs.
• Memory: append-only SQLite — monotone, inspectable learning.

Verify it in 60 seconds (no credentials)

The default provider is the deterministic mock adapter, so every decision path — all five verdicts and the learning loop — reproduces offline with no API key:

pip install -e ".[dev,mcp]"
python scripts/run_demo.py         # all 5 scenarios + the day1 -> day2 learning loop
python -m pytest -q                # 101 tests creds-free (103 with a DashScope key)
python scripts/mcp_client_demo.py  # a real MCP client invokes the probe tools over stdio

Evidence you can read without running anything: the saved MCP stdio round-trip (the probes are not just exposed as MCP, they are called), a captured live qwen-vl-max run, and exactly how Alibaba Cloud is used.

Challenges

The honest hard part was correlated failure: the text and vision legs share weights, so two confident-but-wrong LLM legs can agree. We resolved it by making the deterministic probes the anchor and treating "real but unprobeable" dangers as escalations rather than letting the model talk itself into paying. The other was cold-start — day one has no memory — solved by shipping the 11-mode catastrophe registry as a floor so the agent is paranoid from the first payment.

Accomplishments

A decision engine where every path is reproducible offline (101 tests on the mock provider, creds-free; 103 with the two live qwen-vl-max tests), a genuine learning loop you can watch (day 1 pays a miss, a human teaches it, day 2 catches the same failure), and a multimodal falsification check that is structurally beyond a text-only agent.

What's next

Value-of-information ordering (buy cheap evidence before expensive evidence and stop on EV), deeper document probes (line-item three-way match, duplicate-across-vendors detection), and a durable cross-instance memory store.

Built With

• alibaba-cloud
• dashscope
• fastapi
• mcp
• python
• qwen-cloud
• qwen-max
• qwen-vl-max
• sqlite