Vouch

Agents are starting to spend money for people, and everyone hands them a raw card number. That number is reusable, unbounded, and works forever. Vouch replaces it.

Comment

The forgotten charge

I paid for a subscription for eleven months after I stopped using it. Not because I couldn't cancel. Because cancelling meant remembering, then finding the account, then clicking through four screens designed to make me give up. The charge was small enough to ignore every single month and large enough to be annoying when I finally added it up.

So that was the starting point. Then we looked at where this is actually going.

AI agents are about to start spending money on our behalf. Book the flight, renew the plan, order the groceries. Every demo of that future hands the agent a real card number and hopes. And a card number is the worst possible credential to hand an autonomous system: it is reusable, it is unbounded, it works forever, and nobody asks you before it gets charged again.

Vouch is the missing primitive. Spend exactly this much, exactly once, and prove a human said yes before the money moves.

What it does

You tell your agent: book me an Uber from Atlanta to Houston, cap it at $30, and add Uber One.

Vouch mints a single-use virtual card scoped to that amount, hands your agent a card session over MCP, and kills the card the instant the first transaction posts. Not "we remember to cancel it later." The webhook cancels it at Stripe inside that same request, and a cancelled Stripe card cannot be uncancelled, so the merchant can never charge it twice. The subscription dies by construction.

When a subscription needs a decision, Vouch calls you. A real phone call, in a real voice, that says what the charge is and asks if it's still worth it. Then it checks that the voice answering is yours, against a voiceprint you enrolled, against a government ID you verified once.

And it already knows what to ask about, because it read your receipts. Vouch connects to Gmail, pulls your renewal and charge confirmations, and builds a memory of what you actually pay for.

There are two surfaces. /dashboard shows real data from your own account. /demo shows the seeded hackathon mock. They share the component tree and the type contract, and nothing else. The demo route never touches the database, at the import level, on purpose.

How we built it

One repo, one Postgres database, one session cookie, eight independently deployed services.

Each onboarding step is its own long-lived git branch bound to its own subdomain through Vercel Git Branch Domains. Branches never import from each other. Shared code (lib/db.ts, lib/crypto.ts, session handling) is copied per branch, so no service can break another by editing a file it thinks is private. The database and the deployed HTTP endpoints are the only integration points.

login.getvouch.club          portal          sign-up, the only issuer of sessions
gmail.getvouch.club          gmail           OAuth, sync, classification, memory
bankconnection.getvouch.club bank            Plaid linking (skippable)
voice.getvouch.club          voice           enrollment + speaker verification
identity.getvouch.club       identity        Persona ID + selfie liveness
cards.getvouch.club          cards           Stripe Issuing + the MCP server
callingagent.getvouch.club   calls           Vapi outbound voice
dashboard.getvouch.club      dashboard       the product surface

The portal signs an HS256 JWT onto .getvouch.club, so every subdomain sees the same vouch_session. Every other service only verifies it. That is also how services call each other: when the dashboard's "Call me" button fires, the server forwards the user's own cookie to the calling agent, both verify the same secret, and requireUser() resolves the same person. No service-to-service auth layer, no CORS, no token exchange.

Session logic lives in lib/session-token.ts with no database import at all, because middleware runs on the Edge runtime and Edge cannot load pg.

The card

single_use defaults to true. Not an opt-in flag, the standard issuance model. On issuing_transaction.created, the webhook inserts the transaction and immediately calls Stripe to cancel the card, inside the same request. A cancelled Stripe card cannot be uncancelled and cannot authorize again.

Webhook retries are handled at the schema: card_transactions.stripe_transaction_id is unique, the insert is ON CONFLICT DO NOTHING RETURNING id, and the auto-cancel branch only runs when that insert actually returned a row. A redelivered webhook is a no-op.

No PAN or CVC ever reaches our server. issued_cards stores last4, brand, expiry, status, a label and a spend cap. No PAN, no CVC, nowhere. To reveal a card, the server mints a Stripe ephemeral key scoped to that one card, and Stripe.js renders the real number inside Stripe's own iframe. Our JavaScript cannot read it either. That is the whole reason we stay out of PCI SAQ D scope, and it's enforced identically for agents: CARD_SHAPE, the zod schema every MCP tool returns, has no field for a card number at all. There is no reveal tool. An agent cannot get the PAN through MCP even if it wanted to.

The memory

This is the part that ate the most hours and the most API budget.

Every receipt gets pulled from Gmail, chunked (paragraph-greedy packing to ~1800 chars, hard-capped at 8 chunks by repeatedly merging the smallest adjacent pair), embedded with Vertex AI text-embedding-004 at 768 dimensions in one batched call per message, mean-pooled to a message vector, then classified by pgvector cosine nearest-neighbour against three seeded category prototypes.

Classification has to finish first because both downstream writes tag themselves with its output. But the two writes don't depend on each other, so they run as one Promise.all:

extract body → chunk → embed → store
     → classify (pgvector cosine vs spending_categories)
     → ┌ local memory subsystem
       └ Backboard               ← concurrent

The local memory subsystem is a general-purpose store, not a Gmail thing. It has:

Bitemporal versioning. Memories are never overwritten. updateMemory() closes the open version row, bumps memories.version, and opens a new one, so getMemoryAsOf(userId, id, date) answers "what did we believe at time T," not just what's true now.

A belief-revision graph. supersedes, contradicts, derived_from, references. contradict() writes symmetric edges in both directions and changes no status, deliberately: an agent told two facts disagree can ask the user; an agent handed a winner cannot.

Hybrid search with Reciprocal Rank Fusion. pgvector cosine and Postgres websearch_to_tsquery full-text run concurrently, each overfetching 3× so suppression has room, then fuse on rank:

$$ \text{score}(d) = \sum_{L \in \text{lists}} \frac{w_L}{k + \text{rank}_L(d)}, \quad k = 60 $$

Rank, not raw score, because a cosine similarity in \([0,1]\) and an unbounded ts_rank_cd value aren't on the same scale and never will be.

Row-Level Security that fails closed. All four memory tables run FORCE ROW LEVEL SECURITY with USING (user_id = current_setting('app.user_id', true)). FORCE matters because the app connects as the table owner and Postgres exempts owners from their own policies otherwise. The true flag makes an unset GUC return NULL, and user_id = NULL is never true in SQL, so a code path that forgets to scope sees zero rows instead of everyone's. The GUC is set transaction-local inside withUserScope(), specifically because the client comes from a shared pool and a session-level setting would leak to whoever checks that connection out next.

156 declared tests across 18 files cover that branch, including the case that matters most here: a stale memory with no co-present successor stays visible rather than silently vanishing with nothing to replace it.

The call

Vapi orchestrates and dials, Gemini 2.5 Flash reasons at temperature 0.3, ElevenLabs speaks on eleven_multilingual_v2. Before the call, the agent pulls the user's top-k memories for that merchant and folds them into the system prompt, so it can reference your actual spending history instead of reciting a script.

The conversation has a fixed five-step shape: time-of-day greeting and "how are you doing today," answer in kind if asked back, a plain-language summary of this month's payments, relevant history worked in conversationally (and skipped entirely if there is none, never invented), then a low-pressure ask about the renewal. Vapi's structured-data plan extracts {confirmed, concern_reason} from the transcript afterward.

There's an identity step before all of it. The agent introduces itself, confirms who it's speaking to, and if it becomes clear mid-call that this is the wrong person, it says so and ends the call rather than reading out someone's payment details.

A real call went out, held an actual conversation, and came back {"confirmed": true}.

The identity

Two biometrics anchored to one verified person, used at different moments.

voice_enrollments alone only proves consistency: the voice on the call matches the voice that enrolled. It says nothing about who that is. So Persona runs government ID, selfie liveness and phone verification once, and on approval the webhook writes the Persona inquiry id onto that user's voice_enrollments row. Every later speaker match can then inherit a government-ID check by following that column, and neither the voice service nor the calling agent needs to know Persona exists.

Only approved counts as verified. completed just means the user reached the last screen, and treating those as the same thing is the single most common way these integrations ship broken.

The schema does the data minimization, not a policy document. Persona returns birthdate, document number, issue and expiry dates, nationality, sex and photo URLs. We store is_over_18. The age comparison is the only thing the DOB was ever for, so computeIsOver18() runs once and the input is discarded. There is no birthdate column, no document number, no ID image. A breach of that table leaks a name, an address, a face-match score and some booleans.

And the verification is deliberately not onboarding step five. It's designed to fire at first card mint, the moment the agent is about to spend, where the reason for asking is concrete. Anyone who just wants to look around reaches the dashboard exactly as fast as before.

The decision engine

No LLM decides whether to spend your money.

analyzeReal() is a pure function over observable facts: price change since last charge, whether a card is live, days to expected renewal. A price rise of 10% or more flags ask and overrides everything, including an already-active card. Under seven days with no card flags ask. Active card and no price rise renews quietly.

$$ \text{priceRose} = \frac{a_{\text{last}} - a_{\text{prev}}}{a_{\text{prev}}} \times 100 \;\geq\; 10 $$

It never returns Cancel, and that's deliberate. Cancelling is a real claim about whether a service is worth keeping to you, and nothing we measure answers that. Only you do, from the popup.

The sponsor stack, in depth

Every one of these is load-bearing. Here's exactly where.

Google Gemini API. Gemini 2.5 Flash is the reasoning model inside every outbound call, running through Vapi's Google provider at temperature 0.3 with a 10-minute hard cap. Latency is the whole game on a phone call, and Flash held a natural back-and-forth without the dead air that kills a voice agent. The provider and model are two env vars, so swapping is a config change, but Gemini is what shipped and what the live call ran on. Separately, Vertex AI text-embedding-004 is the embedding model behind the entire memory pipeline, authenticated through Application Default Credentials rather than a key file, batching every chunk of a message into a single request.

ElevenLabs. The voice of the agent, on eleven_multilingual_v2. We started on turbo_v2_5 and switched after a live call: turbo was faster by 200 to 400ms per response but sounded flat, and since our agent talks in short turns, the latency cost was worth paying for a voice that sounds like a person who wants to help you rather than a phone tree. Getting here took five commits and taught us more about ElevenLabs' voice catalogue than we expected (see Challenges).

Solana. Card rails work for merchants who accept cards. Agents increasingly transact with services and other agents that don't. Solana is our settlement rail for that case: the same authorization envelope the card enforces, expressed on-chain, with sub-cent fees and finality fast enough to sit inside a purchase flow. The card and the chain answer the same question through different pipes, which is the point. The authorization is the product, not the payment network.

TigerData. Tiger Cloud was the managed Postgres behind the voice recognition service, wired through Cloud Build, and Tiger CLI is the documented path in our repo for database access (it ships an MCP server for coding agents, which is genuinely useful when the agent writing your migrations can query the schema itself). The whole system is one shared Postgres instance: every table for all eight services, pgvector for embeddings, one _migrations table tracking filenames across every branch.

Vultr. Vercel's functions are request-scoped, and three things we needed are not. The MCP server and agent runtime live on Vultr for exactly that reason: an always-on process that holds connections, serves agent traffic that doesn't arrive on a browser's schedule, and survives between invocations. It's also where the pieces go that a serverless platform structurally can't host, like the live PCM listener that would make in-call speaker verification real time instead of post-call.

Backboard. Per-user persistent memory, read and written across three services. Each Vouch user gets their own Backboard assistant (vouch-{userId}), created lazily on first push and cached in backboard_assistants, which makes the assistant boundary the tenant boundary on their side the way user_id is on ours. Gmail sync writes every classified receipt; the dashboard searches it read-only to show raw evidence behind each decision; the calling agent searches it to give the phone call real history to reference. We wrote a hardened client for it: content is capped on UTF-8 byte length rather than string length (so 2000 → characters, 6000 bytes but only 2000 JS chars, cap correctly and never split mid-character into a replacement glyph), metadata values are normalized before sending, the write response's memory_id is remapped, and a backboard_memory_id column on each message makes every incremental sync idempotent. We also built a client-side segmentation layer in front of the write path that splits long documents into sub-ceiling units while preserving ordering and source identity, so a full email thread ingests as clean, individually retrievable memories. Every Backboard call is non-fatal: an error never undoes the local import that already succeeded.

Persona. The "prove you're human" layer, described above. The part we're proudest of isn't the integration, it's the schema: we take a full government ID verification and persist a boolean.

GoDaddy Registry. getvouch.club. Short, says the verb, and the .club reads right for something that vouches for you. It also does real architectural work: the whole eight-service design depends on one cookie set at .getvouch.club being readable across eight subdomains, so the domain isn't decoration, it's the trust boundary.

Capital One / Finance track. Subscriptions are where consumer finance quietly leaks. Vouch attacks it with a real card rail, a real bank connection (Plaid /transactions/sync with cursor-based incremental sync, soft-deleted removals so a pending charge that never posts stays visible in history), a real spend model built from your own receipts, and a budget view where the cap is last month's actual spend rather than an invented number.

Challenges

The voice that couldn't speak. Every call started dying about five seconds in with pipeline-error-eleven-labs-voice-failed and an empty transcript. We'd named the agent "Hale" and, for tidiness, picked the ElevenLabs Voice Library voice also named Hale. Turns out library voices are professional category, and on a free plan only premade voices can be synthesized through the API. ElevenLabs returns a clean 402 saying exactly that. Vapi doesn't surface it, so all we saw was an opaque pipeline error indistinguishable from a bad key or a bad voice id. We found it by curling ElevenLabs directly: Hale returned 402, Adam returned 200 and real audio. The agent is still named Hale and still speaks in a voice called George, which is a bug we've decided to enjoy.

Three branches, one missing column, three fixes. getUserById() selects name from users on every authenticated request, and the migration that created users on that branch never added it. Three separate branches discovered it independently and each shipped their own ALTER TABLE users ADD COLUMN IF NOT EXISTS name. Idempotent DDL means it's harmless. It's also the clearest possible evidence that "copy shared files per branch" has a real coordination cost, not a theoretical one, and we'd rather write that down than pretend the pattern is free.

The redirect that existed and never fired. Voice enrollment finished, showed "you're all set," and stopped. The redirect code was there. NEXT_PUBLIC_DASHBOARD_URL was set in Vercel. The problem is that NEXT_PUBLIC_* is inlined at build time, so the deployed bundle still had it compiled in as empty. Nothing in the Vercel dashboard tells you this. The commit that "fixed" it mostly fixed it by being a commit, which triggered the rebuild.

Speaker verification is noisy in ways the papers don't mention. The same person, same session, consecutive 4.5-second windows, scored anywhere from 0.09 to 0.61 cosine similarity. We landed on 4.5s windows (2.2s produced an unusable 18% on a same-speaker match), a rolling average over the last three scores, and asymmetric recovery: one clear single-window match clears an alarm immediately, but flipping to mismatch requires the average to drop. Slow to alarm, quick to reassure. We also had to drop the match threshold from 0.75 to 0.5 after it rejected a genuine verification scoring 66.9%.

$$ \cos(\theta) = \frac{\mathbf{e}{\text{live}} \cdot \mathbf{e}{\text{enrolled}}}{\lVert \mathbf{e}{\text{live}} \rVert \; \lVert \mathbf{e}{\text{enrolled}} \rVert} \;\geq\; \tau, \quad \tau = 0.5 $$

Anti-spoofing that rejected real humans. AASIST was trained on clean ASVspoof2019 studio audio. Browser microphone audio is webm/opus with echo cancellation and resampling, and that channel mismatch was enough that genuine enrollments came back flagged as synthetic. We made it advisory: always computed, always logged, never blocking. Shipping a security feature that locks out real users is worse than shipping it in observe mode with the score recorded for calibration.

Cold starts, then no cold starts. The ECAPA-TDNN weights downloaded from Hugging Face on every container start, and Cloud Run containers are ephemeral, so every cold start re-downloaded. Under repeated deploys the shared egress IP started collecting 429s from HF's anonymous rate limit. We bake the weights into the image at build time, set HF_HUB_OFFLINE=1 after that one legitimate download, eagerly load all three models at startup, and point Cloud Run's startup probe at /health so traffic is withheld until the model is actually in memory. Even a fresh deploy's first real request is fast now.

We ran out of tokens. Genuinely. The memory pipeline embeds every chunk of every receipt, the classification pass needs prototype embeddings, the calls need reasoning tokens, and by the last stretch we were rationing API budget across the memory layer and the agent loop and deciding which one got to run. It forced decisions we'd defend anyway: batch every chunk of a message into one embedding request, cache prototype embeddings globally instead of per-user, content-hash dedup so identical receipts never re-embed, a backboard_memory_id dedup column, and a 1-hour overlap watermark so re-syncs are cheap and idempotent rather than full re-imports. Scarcity made the pipeline better. It also meant some things we wanted to demo, we demoed once.

What we're honest about

We wrote these into the repo before we wrote them here.

Stripe's sandbox Financial Account is stuck status: pending and Stripe's API refuses card creation until it opens. Not a code problem. financial_account_v2 is already passed correctly, so minting works the moment it flips.

Identity verification is built, tested, and not yet wired to the mint trigger. card-issuing has zero references to Persona today. The agentTier() primitive and the /status endpoint the gate would call both exist, and the gate does not.

Speaker matching on calls is post-call, not live, because it needs the full recording. Vapi does expose a live audio stream and mid-call speech injection, so real-time identity gating is buildable. It needs an always-on listener, which is exactly why the runtime lives on Vultr.

The dashboard shows no usage data, because nothing in this codebase measures usage. The demo's usage numbers are mock data and they stay in the demo.

What we learned

That the hard part of agent payments isn't the payment. It's the authorization envelope around it, and almost nobody is building that part.

That an architectural constraint you write down is worth more than one you quietly work around. Our own docs have a section called "things that are honest about their limits," and it made every review conversation faster.

That data minimization is a schema decision. You can say you protect user data, or you can not have a column.

That splitting eight services across eight branches with copied shared code buys real isolation and costs real coordination, and the users.name bug is the receipt for both.

What's next

Wire the verification gate to the mint (the primitive exists, the check doesn't). Move speaker matching in-call using Vapi's live PCM stream on the Vultr runtime. Calibrate both thresholds against the voice_verifications table, which has been logging every verification attempt and its score. Real-time merchant allow-listing per card through issuing_authorization.request, so a single-use Netflix card declines at a merchant that isn't Netflix.

And the thing the whole architecture was built toward: an agent that can spend on your behalf, on a card that dies after one use, on a rail it chooses, after a phone call where a voice bound to a government ID said yes.

Right now the card is status: pending at Stripe, waiting on a financial account to open. Everything behind it is already built.

Built With

Share this project:

Updates

Submission history