Withheld

• 

Inspiration

Every errand ends the same way: "we'll call you back." For most of us that is mildly annoying. If you are Deaf, non-speaking, or cannot hold a spoken conversation with a stranger at an unannounced moment, that sentence does not delay the errand. It ends it.

The gap is specific, and it is not "AI can't talk on the phone." Text relay services already exist. The problem is the callback: an unscheduled call, from an unknown number, at a time nobody agreed to, with no text fallback. A clinic with no online booking, a garage, a landlord, a government office whose web form has been "temporarily unavailable" for two years. If you cannot answer a phone, a callback is not an inconvenience. It is a door that does not open.

So we stopped trying to make the person answerable, and made the errand not require it.

What it does

Withheld is an errand engine built on CALL-E. It places the call, and when the other side offers to ring back, it declines and asks for a time window instead, then calls back itself inside that window. The person never answers a phone at any point, and the business never has a number to ring.

One sentence is the whole product: a callback is a door you cannot open, so never accept one.

Three things follow from that:

The number is not withheld by asking nicely. It is never put in the model's reach. The result schema handed to CALL-E has no field for a phone number in either direction. Asking a model not to say something is a request. Giving it nowhere to say it is a design. Every transcript is then re-scanned afterwards anyway, including numbers spoken in words ("oh double seven oh oh"), and a leak is reported, never hidden.

Every answer carries the quote that produced it. CALL-E's own verdict is recorded and read by nothing. If the provider says the booking succeeded and no turn in the transcript says so, the report says "not answered." A window counts only if a callee actually named a time, and only inside hours the person set. "Sure, that's fine" names nothing and confirms nothing.

It says "unproven" more often than "yes." Nobody picking up is not the clinic declining. An unfinished call yields no verdict. When the callee demands a number and the agent has none, holding the line is the success, and the errand goes to a human rather than being dressed up as progress.

Everything comes back in writing, because the person could not hear the call.

How we built it

TypeScript core, Next.js console, CALL-E for the calls. Both CALL-E surfaces are integrated behind one CallProvider interface: the developer SDK (@call-e/calle) and the MCP CLI (@call-e/cli). We shipped both because the published installation guide ends at calle auth login, which authorises the CLI and never produces an API key, so an SDK-only integration is unreachable for most people following the official path.

The chain is the non-obvious part. A callback offer normally terminates an errand. Here it advances it: leg one extracts a window bound to its supporting turn, leg two is proposed into that window, and the chain never schedules a call by itself.

Fixture replay is the default everywhere. The full test suite, the entire CLI, and the whole web console run with no credential, no network, and no calls. --real is the only path to a phone ringing, and it refuses without a usable credential.

Challenges we ran into

We leaked the number in our own app. Next.js serialises every prop passed to a client component into the RSC payload, including fields never rendered. Our phone number was sitting in the page source of a product whose entire premise is that it never gets out. The fix was to build the client object field by field and never by spread, locked by 14 tests. Finding that in our own code, in the one place we claimed to be careful, was the most useful hour of the build.

Two official CALL-E packages install the same binary name. @call-e/cli and the SDK @call-e/calle both install a command called calle, and they share no commands. With the SDK as a project dependency, anything that puts node_modules/.bin on PATH silently gets the wrong one, and every command fails with Unknown command: auth, which reads like a broken login. We now resolve the binary ourselves and skip node_modules.

The docs and the live schema disagree, silently. After authenticating, we read the real get_call_run output schema back from the MCP endpoint and diffed it against the CLI reference. Statuses are spelled NO ANSWER, not NO_ANSWER, so an unrecognised terminal status reads as "still ringing" and the poll loop would have run to timeout on every unanswered call. The transcript is at result.transcript, not transcript, and reading the wrong key returns null, which is indistinguishable from a call where nobody spoke, so every errand would have been reported as a no-answer. Both are fixed and covered by tests. We found them without spending a single call.

We found we were not first. Partway in we discovered apps/typescript/call-on-behalf, already merged in this repo, covering consent-scoped disclosure with concepts close enough to ours to be uncomfortable. We verified by grepping the whole repository that the callback refusal and the multi-leg chain were genuinely unclaimed, then rewrote our prior-art section to credit it plainly rather than quietly compete with it.

Accomplishments that we're proud of

The refusal itself. Watching the agent get offered a callback and decline it, then ask for a window instead, is the moment the idea stops being a slide and becomes a mechanism.

Designing the number out rather than guarding it. There is no field to put it in.

131 tests, and a demo anyone can run and a judge can click, with no credential and no phone ringing. The hosted console at https://withheld.vercel.app is fixture-only by construction. We verified zero number leaks across all six scenarios in production, not just locally.

Being willing to say what does not work. The Limits section and the honest prior-art credit cost us nothing we wanted to keep.

What we learned

Fail closed is not a slogan, it is a series of small asymmetric bets. When the CLI hands back an unlabelled transcript line, we refuse to attribute it, because the two ways of being wrong are not symmetric: wrongly crediting the callee could confirm a booking nobody offered. When a submission might already be in flight, we stop rather than retry, because re-dialling a real person "to be safe" is not safe.

Also: never trust a provider's self-assessment, including a good one. And read the live schema, not the docs.

What's next for Withheld

Real calls at volume against the three errand types we modelled (clinic booking, landlord repair, government appointment), to find where the callback refusal actually fails in the wild and how businesses respond to it.

A first-class handoff for the needs_human path, because that is the honest outcome and it currently just stops.

Composing properly with call-on-behalf rather than sitting beside it, so consent scoping and callback conversion are one pipeline.

The feedback we owe CALL-E: eight reproducible findings, including the binary collision, the schema mismatches, and a login session that expires in 25 minutes while the CLI polls it forever with no er

Built With

• call-e-mcp-cli
• call-e-sdk
• css
• next.js
• node.js
• react
• typescript
• vercel
• vitest