No Undo

Inspiration

We had planned a different project with different sponsors. Then Foxit published its challenge, "Your Agent Shouldn't Sign That," and it described something we had already half-built.

Foxit's open-source MCP server exposes about 40 tools for document work, and every one of them is reversible. Signing is deliberately left out. An agent that wants to send a document for signature has to cross a boundary into the eSign API, and the design of that crossing is what gets graded.

We had already published an npm package for exactly that shape of problem, safe-write-mcp-core: propose, preview, approve, execute. So before writing anything new we reread our own core looking for the weak point, and found one.

consume() flipped a plan token to used, wrote an "executed" audit record, and returned. The host made the real API call somewhere the core could not see. Both orderings fail under a crash. Consume first, and the audit log claims a document was sent that never went out. Call first, and a restart re-sends it, because the plan store lived in memory. Our own design notes defended that in-memory store: a restart invalidates pending plans, which is the safe direction. That reasoning is correct for a database you can roll back, and exactly backwards for an API where sending is final. With rollback, the dangerous window sits before the call. Without it, the window straddles the call, and the core did not model that at all.

That bug became the project. Everyone at a hackathon demos a happy path. We decided to demo a crash.

What it does

One prompt goes in, a signed document comes out, and exactly one step in the middle stops for a human.

Take this freight invoice, redact the PII, and send it to Alice and Bob for signature.

That line drives seven steps:

1. A parser turns the sentence into a typed payload: folder name, recipients, document source, instructions. The parsed fields go back onto the approval card, so a human can catch a misread before anything irreversible happens.
2. Foxit's MCP server assembles the invoice, HTML through pdf_from_html and get_task_result.
3. Nutrient DWS /extraction/extract reads the source document, and a router sorts every field: match label first, confidence second, grounding and OCR recognition floors as vetoes. The card reports how many fields cleared, which ones the recognition floor caught by name, and which came back ungrounded.
4. Nutrient /build applies the PII redactions, then /extraction/parse reads the redacted document back and confirms each target value is gone and each signature field survived. A redaction we cannot prove kills the run before a draft folder exists.
5. The gate. The agent renders the plan (prompt, recipients, extraction routing, what was removed plus the proof it is gone, the document SHA-256, and a plain irrevocability warning) and waits. The approval server binds an ephemeral loopback port and requires a per-session bearer token on every route.
6. Approval releases the one irreversible call. The eSign API is hit directly, not through MCP, with a durable ledger keyed on the plan token and a folderStatus reconciliation (DRAFT versus SHARED) that makes the send happen once.
7. The adapter polls until folderStatus == EXECUTED and downloads the signed PDF.

Everything before step 5 is reversible and runs unattended. That is the whole argument: signing stays outside the MCP catalog because the catalog's value is that nothing in it can hurt you, and adding a signing tool would spend that.

Here is the crash, verbatim from the recording, against the live API:

$ NO_UNDO_CRASH_AFTER_FSYNC=1 node agent/esign-agent-loop.mjs --auto-approve --doc messy \
    --prompt "Take this freight invoice, redact the PII, and send it to Alice and Bob for signature."
[agent] Draft: folderId=35704250 planToken=7c61234a...
[esign-audit] executing token=7c61234a... tool=esign_send
[crash-injection] SIGKILL after beginExecute fsync (token=7c61234a...) before the gateway send

$ node agent/esign-agent-loop.mjs --auto-approve --doc messy --prompt "Take this freight invoice..."
[agent] Recovered 1 stuck-executing plan(s) (reconciled on load):
  - planToken=7c61234a... folderId=35704250 -> folderStatus=DRAFT
    -> confirmed not executed -> released for retry

The kill landed on the DRAFT side of the window, so the gateway had never been called and the plan was released instead of left stuck. Had it landed on the other side, the same query returns SHARED and recovery records the send rather than repeating it. The system of record picks the branch. We never guess, and no plan is ever sent twice.

How we built it

Node 18, no framework. The parts that matter:

The core, as a separate published package. safe-write-mcp-core owns the plan lifecycle and nothing else. We split consume() into beginExecute() and confirmExecuted() / confirmFailed(), so "executed" is only audited after the host confirms. beginExecute() writes executing to an append-only journal and fsyncs it, parent directory included, before the gateway call happens. A process that dies mid-send leaves a plan in a state you can query instead of one that got forgotten. Reconciliation is a host-supplied callback returning done, not-done, or unknown, bounded by a timeout so a hanging vendor cannot stall recovery. Foxit's implementation of that callback is the folderStatus check. The core shipped 0.4.0 mid-hackathon to add the approval-server bearer token, and this repo consumes it by version.

Two MCP servers and one deliberate hole. Foxit PDF Services drives assembly over stdio. A second server exposes the Nutrient stage. The eSign send is the only call that does not go through MCP, and that is the point.

Fixture seams from day one. Our agent sandboxes could not reach the vendor hosts, so every adapter got a recorded-transcript seam before it got logic. Each stage ships a one-command probe that makes the live billed call and commits its response. The 283-test suite (48 suites) makes no network calls and needs no credentials. Every extraction claim in the repo points at a committed API response, including the ones that make us look bad.

Hash-chained audit. Every decision appends to an fsync'd JSONL trail where each record carries the previous record's hash. Editing one invalidates everything after it.

The whole thing is 95 commits and 63 pull requests across 16 days, each one through CI and an automated review round before merge.

Challenges we ran into

The extraction API lied, confidently. On the first live call, Nutrient's understand mode read Total due $86.86 and returned total_amount: 26.86, at match: id_match, confidence: 0.970, groundingScore: 0.95. Every signal a reasonable integrator thresholds on said auto-approve. Only recognitionScore (0.678) dissented, because it is the only one measuring whether the glyphs were read, rather than whether the value came from the right region of the page. We tried agentic mode next, which costs double. It got the totals right, quietly mangled the line-item table instead, and emits no recognitionScore at all. Paying more bought better answers and less ability to tell whether they were right.

The redaction API lied in the opposite direction. preset: "vin" was on our confirmed list because it returned HTTP 200 when probed. It does return

1. It also does not redact. Against a document containing the VIN 1FUJGLDR8CLBP8834, one target at a time:

Target	HTTP	Output	VIN after apply
preset: "vin"	200	valid PDF, 67,142 bytes	still there
regex: "[A-HJ-NPR-Z0-9]{17}"	200	valid PDF, 69,467 bytes	gone

Both calls succeeded. Both returned a document that opens. The only difference a caller can see is a byte count, and nobody thresholds on byte counts. Our list had recorded which identifiers the API accepts, which turns out to be a different question from which ones match. This is why the pipeline reads the redacted document back instead of trusting the apply call.

The intermediate artifact was a trap. The staged redaction returns a PDF that renders black boxes over the PII. All five planted values are still recoverable from it with one API call. An operator who eyeballs that file, sees the boxes, and forwards it has published everything they believed they had removed. We keep docs/fixtures/probe-staged.pdf in the repo precisely because it is unredacted. It is evidence, not an oversight.

Redaction boxes ate the signature fields. The first assembly put each signer's email next to their Foxit Text Tag. Redacting the email destroyed the tag, and the gateway refuses a document with no signature field. The fix was to stop putting signer addresses in the document body at all, since Foxit already carries them in parties, and to assert tag survival in the same verification step that checks the PII is gone.

Our audit log recorded a send that never happened. The gateway returned HTTP 200 with an error body, and we believed the status code. The send is now verified against the system of record before anything is written down.

Our own crash demo was wrong. The first version injected the kill before the send instead of inside the dangerous window, and recovery ran silently, so the recording proved nothing. We filed it as a P0 against ourselves and rebuilt the beat.

A vendor engine drifted under us, and we caught it with hours left. On Sept 3 we reran the calibration probe against the same generated invoice. understand mode returned the same two wrong dollar amounts as three weeks earlier, with identical match labels, confidence, and grounding. The recognitionScore had moved: 0.678 to 0.834 on the total, 0.569 to 0.842 on the tax. Three consecutive runs returned byte-identical scores, so this was a new stable snapshot, not request noise. At the recognition floor we had shipped (0.8), the old snapshot is caught and the new one auto-approves both wrong numbers. The property test that should have caught this was picking the newest fixture by filename, so it stayed green until we pointed it at the new file, at which point it failed against our own claim that no field disagreeing with the document is ever auto-approved. We raised the floor to 0.9 and wrote down the uncomfortable part: the best recognitionScore we have ever seen on a correct field is 0.859, and the worst on a wrong one is 0.842. That is a 0.017 margin, smaller than the drift we just watched happen.

Accomplishments that we're proud of

The crash demo works, on the live API, and the recovery branch is chosen by asking Foxit rather than by trusting local state.

Verification catches a real failure, not a hypothetical one. We can point at a redaction call that returns 200 and does nothing, and at a pipeline that refuses to send because of it.

The approval card admits what it does not know. It prints thresholds calibrated: false because that is true. It names which document extraction actually read, because "0 of 16 fields auto-approved on a clean render" and "0 of 16 on a bad scan" mean very different things about how much judgement was exercised. It describes the regex redaction target by shape and never prints the pattern, because a redaction pattern encodes the values it hides, and putting it on the review screen would leak them.

283 tests, no live calls, no credentials required to run them. Every claim in the README is backed by a committed response.

We shipped upstream instead of patching locally. The approval server had no shared secret, which we had listed as a known gap. Core 0.4.0 added a per-session bearer token on every route, and this repo consumes the published version.

And we said no to a feature at the end. Real N-of-M approval needs approver identity in the token record, idempotency keyed on who approved, and per-approver auth. A shallow local version would let one person satisfy a 2-of-3 rule by clicking Approve twice, which is worse than not having it. It is in the plan, traced, and cut.

What we learned

HTTP 200 means the request was well-formed. It is not evidence that anything happened. Two endpoints from the same vendor taught us this in opposite directions inside two weeks.

The signals that tell you an action is safe are not the signals that tell you it is correct. Confidence, match label, and grounding all agreed on a number that was wrong by sixty dollars. A gate that thresholds on the safety signals and calls it verification is decoration.

Vendor models change under you with no version marker and no changelog. Our thresholds were tuned against a snapshot, and the snapshot moved. The only defense we found is committing raw responses with dates and rerunning the probe before making claims.

The dangerous window straddles the call. Most retry designs put the durability boundary on one side of it or the other, which is fine when the operation is reversible and wrong when it is not.

Writing down the gaps is cheaper than defending them later. Every known gap in this repo was found by us, in writing, before anyone asked.

What's next for No Undo

Calibrate the thresholds properly. Every entry is still marked calibrated: false, and a test asserts none of them claims otherwise. Doing it right needs a representative corpus per document type, not one invoice. The Sept 3 drift makes a stronger case than that: a static floor may be the wrong shape entirely, and a relative signal or a cross-mode agreement check might survive engine changes better.

Replace the fixed redaction list with a detector. Today it covers email, North American phone, and a VIN regex, because those are what this invoice carries and what we probed to actually match. A passport number or an IBAN passes straight through, and verification reports success, because it only confirms that what it was told to remove is gone.

N-of-M approval, at the right layer. Approved plans need a recorded set of approvers instead of a boolean, idempotency keyed on identity, and per-approver auth in the core.

Reconcile the redaction step. eSign can answer "did this happen?" because a folder has a status. Nutrient's /build has no job ID and no server-side state to query, so a crash mid-apply leaves reconciliation returning unknown. Honest, but a human still has to look.

Wire the second irreversible action. The gate, the durable plan store, and the exactly-once ledger know nothing about documents. eSign is just the first thing plugged into them. Payments, production deletes, and outbound messaging have the same shape, and the same industry answer today: a confirmation dialog that does not survive a crash and cannot tell you whether the thing you are approving is correct.

Built With

• ai-agents
• anthropic
• audit-log
• claude
• foxit
• foxit-esign
• foxit-pdf-services-api
• human-in-the-loop
• mcp
• model-context-protocol
• node.js
• npm
• nutrient
• nutrient-dws
• pdf
• rest-api
• sha-256
• typescript
• vitest