Attestor

An enterprise agent fleet that answers vendor security questionnaires from your own documents, cites every claim, refuses what it cannot support, and holds the rest for a person.

Comment

The email that blocks the deal

Sell software to any company larger than five people and, before they sign, they send you a spreadsheet. Two hundred to four hundred questions about your encryption, your subprocessors, your incident response, your data residency. Every answer needs a document behind it. Every answer has to match what you told that same customer three weeks ago. Then round two arrives.

Large companies employ GRC teams for this. Everyone else loses the week, or loses the deal.

I'm one person. So I built a fleet.

Attestor answers vendor security questionnaires from your own documents, cites every claim, refuses the ones it cannot support, and holds the rest for a human. A customer emails the questionnaire to a watched mailbox. Nobody opens a console. Fifty-one seconds later there is a review in flight.

What it does

Run of record: 150 questions · 136 answered with citations (91%) · 79 checked by a separate agent identity · 63 held for a person · 13 minutes 26 seconds · 613 audit events · $0.14.

An inbox agent classifies the mail and opens a review. Triage splits the questions by department. Three drafting agents — each a separately deployed engine with its own identity and its own corpus — answer in parallel, citing the document, the section, and a relevance score. A verifier, running under a different identity, that did not write the answers and cannot reach the corpus at all, reads each claim against the passages it cites and returns the ones those passages do not carry. Whatever survives is assembled into the customer's own workbook plus a PDF evidence pack, filed to Drive, and replied in-thread.

The most important thing it does is refuse. Nine questions were planted with no supporting document anywhere in the corpus. It refused nine of nine. A fluent, well-cited answer sourced from somewhere other than your own policies, sent under your company's name, is a worse outcome than a blank — so there is a test in the repository whose entire job is to keep this system off the web.

How I built it

Seven engines, seven identities — and the reason is not tidiness. ADK lets you nest sub-agents under one agent. I deployed seven separate reasoningEngine resources instead, because nested sub-agents share a single Agent Identity, and one identity means one service account holding the union of every department's permissions. That is the exact violation a fleet exists to prevent.

The corpus boundary is a credential, not an instruction. The security engine, asked for the legal corpus, does not decline. It is refused — by a conditioned IAM binding, before any model runs:

403 GET storage.googleapis.com/.../attestor-505506-corpus/o/legal%2Fdata-processing-agreement.txt
Caller does not have storage.objects.get access to the Google Cloud Storage object.

The same probe reads its own prefix at 4,298 bytes in the same run. A denial without a matching success proves nothing except that something is broken.

Deterministic orchestration where the sequence is known. Intake → triage → parallel drafting → assembly is a workflow agent, not a model deciding what to do next. The orchestrator is reserved for genuine judgement: which pipeline, which failures to retry, and whether to release the round at all. On the run carrying a hidden prompt injection it held the entire review unprompted — "a guardrail fired repeatedly across multiple questions" — which nothing in the per-answer rules would ever have done.

Confidence is computed, never asked. Citation count, embedding cosine similarity against the retrieved passage, hedging, contradiction against prior commitments. A model's self-reported confidence is uncalibrated and I refuse to ship it as a number a compliance officer might trust.

Durable async, because reviews take weeks. One Pub/Sub message per stage, claimed under a 900-second lease against a 600-second ack deadline — so a redelivery arriving mid-draft finds a live claim and is refused, rather than drafting the same 123 questions twice. Nine redeliveries across the final runs; not one duplicate answer.

Memory that binds. Commitments go to Vertex AI Memory Bank when a round closes. A 37-day dormant review woke on a customer reply, matched a commitment by meaning where ID matching found nothing, and redrafted the contradicting answer under the constraint.

Two observability planes, deliberately. Cloud Trace answers what was slow. An append-only Firestore audit log answers who decided what, on which evidence, under which identity, in six months. A sampled trace store cannot answer a compliance question, and a compliance log is the wrong tool for finding a slow call.

The challenge: nine failures that looked like empty results

This is what I actually spent the build on, and it is the most useful thing I learned.

A read that fails and a read that legitimately finds nothing are different facts. Nine times, something collapsed them. Every one produced a smaller number, a green run, and a confident false statement — and not one announced itself.

Discovery Engine returning [] under a 429 became "the corpus has no answer." A commitment read that caught every exception became "this customer has no prior commitments" — silently disabling the consistency check for an entire round while the citation rate looked fine. An engine that returned fifteen passages at 0.744 relevance and no prose became "no supporting evidence was found in the corpus."

The largest was in Google's platform, not mine. A completed run reported 172 of 312 questions unsupported. I queried the same corpus directly: passages for five of six of them, at 0.950 top relevance. The deployed search was returning empty result sets successfully under load — so the retry never fired, because a call that succeeds with nothing in it is not a call that failed. Retrying empties before believing them moved citations from 43% to 91%.

And my first fix made it worse. The mechanism was a per-minute quota, correlated minute-for-minute against the dispatcher's empties. My retry backoff was nine seconds — shorter than the quota's sixty-second reset — so all three attempts landed inside the same exhausted window. A backoff has to outlast the thing it is backing off from. Widening it past sixty seconds is what recovered the number.

Three more worth carrying out of this build:

A guardrail's sensitivity is a function of window size. The same injection payload was denied alone and allowed when embedded in four hundred characters of ordinary prose. Screening concatenated evidence in one call is not a weaker version of the same defence — it is a different and much worse one. Tool output is now screened in 200-token windows, passage by passage.

A fix that is written, reviewed by eye, and never called is indistinguishable from no fix. The recursive batch-split that handles Model Armor blocking Attestor's own prompts existed for a full phase before I noticed that triage() never called it. Reading call sites, not function bodies, is what found it.

My own security guardrail classified my own security questions as an attack. A batch of forty questions — break-glass access, secrets committed to repositories, national security requests — reads collectively as a prompt injection. Measured boundary: batches of 5, 10, 15, 20, 30 pass; 40 blocks.

What I refused to do

At one point the corpus could only evidence 48% of the questionnaire, and the honest fix was to write the twenty documents a real company of that profile would actually hold. The dishonest fix was to loosen the evidence requirement. I wrote the documents.

Later, the round-two consistency check produced NO_CONTRADICTION on the natural run, because the first draft was already correct. The literal exit criterion wanted a contradiction. Producing one would have meant degrading the corpus until the demo looked better. Instead I planted a "Deployment Options Update" document contradicting a July commitment — which is how this failure actually happens in a company: documentation drifts ahead of what a customer was told in writing. Both results are recorded.

Every number in this project traces to a file in docs/proof/. PROGRESS.md records eleven phases, what was built, how it was verified, and what did not work — including the parts that were marked failing at the time.

Built on

Gemini 3.7 Flash and 3.5 Flash-Lite on Vertex AI · Google ADK 2.7 · Vertex AI Agent Runtime (seven engines) · Agent Identity · Agent Registry · Vertex AI Memory Bank · Model Armor, both directions · Vertex AI Search, one datastore per department · Cloud Run · Pub/Sub + Eventarc · Firestore · Cloud Storage · Secret Manager · Cloud Trace · Gmail and Drive APIs.

Agent Gateway was evaluated and not adopted — it is an L7 proxy whose distinguishing capability is reaching private VPC endpoints, and Attestor has none. Provisioning one would have put a box in the diagram carrying zero traffic. The reasoning, and the condition that would reverse it, are in docs/decisions/ADR-0006.

Eighteen days. 121 commits. 750 tests. Under $20 of the $150 credit.

What's next

Deadline chasing, so a review that has been quiet for a week asks after itself. An answer library that learns from what a human edited rather than approved. And the obvious extension: the same architecture answers RFPs, DPAs, and annual re-certifications, because the shape of the problem — evidence you already have, that someone else needs, in a format that must be defensible — is identical.

The security questionnaire is just the first one that blocks a deal.

Built With

Share this project:

Updates

Submission history