Vouch

Vouch is trust infrastructure for agentic commerce: AI agents investigate refund evidence and build "Vouch scores” for individual shoppers so merchants can identify and resolve credible claims faster.

Comment

Inspiration

Vouch began with three experiences in the days before Cal Hacks. At the airport before his flight to the hackathon, Shubhay tried to resolve an order issue and found that requesting his refund required him to submit a written request followed by a call. Shivam spoke with a single father living paycheck to paycheck who had several hundred dollars tied up in a refund for weeks and his repeated follow-ups brought only "verification pending." Aadi was exploring an agent to simplify airline-ticket refunds and became familiar with the trend towards development of agents to complete these tasks.

Together, these experiences made us ask what personal finance will look like when agents can buy, document problems, and request refunds for us. We decided that merchants need a signal they can trust while consumers needed a faster way to verify. We chose refunds as a focused starting point: simplify the consumer's work while helping merchants detect fraudulent claims, coordinated abuse, and scalping behavior as agent-driven commerce grows.

The stakes already exist: Appriss Retail's research with Deloitte estimated $103 billion in U.S. losses from fraudulent returns and claims in 2024, while NRF's 2025 survey found 71% of consumers were less likely to return to a retailer after a poor returns experience. Appriss Retail / Deloitte · NRF / Happy Returns

Refunds are our first workflow and the focus of our scope for this project. Portable trust for an economy of people and agents is the larger picture.

What it does

For customers, the product offers an accessible, phone-ready claim flow across product categories and refund reasons. They record the item while explaining the problem verbally, and complete an in-video verification challenge. The claim page tracks 10 processing stages as FFmpeg and faster-whisper prepare the footage, Factory Droid investigates it, and CREAO supplies review and trust judgments. A recorded test completed the review in 155 seconds, using live Droid sessions and a fixture CREAO reviewer. When evidence is incomplete, Vouch can request one additional clip with a specific explanation of what is missing.

For merchants, Vouch prepares an evidence-backed case: what was ordered, what the customer said, what Factory Droid's footage analysis supports or contradicts, the return policy, relevant shopping and return history using the customer's Vouch Score, and a recommendation. Findings link to timestamps and proof clips, making audits and company-specific workflow additions easy. When the investigation supports approval, Vouch immediately submits the refund recommendation into the merchant's decision workflow; recommendations to reject go to the company for human review rather than automatically denying the customer. Eligible approvals can proceed automatically within the configured reviewer and amount checks; payment settlement remains separate, and demo payouts are simulated. For merchants, this means less time spent collecting evidence and reviewing routine claims, and faster resolution for credible customers, reducing churn. It also provides clearer signals for which requests need closer scrutiny, while keeping merchants in control of their policies and decisions.

For trust, Vouch maintains a shopping reliability score built on a customer's historical purchases and returns, evaluating their shopping and return integrity with a recorded reason for each change. CREAO's Refund Investigator and Risk Scorecard contribute to the score judgment; SpacetimeDB bounds the applied change and prevents applying it twice. The planned CRS identity and contact-risk integration will add context about who is submitting the claim, separately from whether the refund itself is valid. An unresolved request for an extra clip does not itself change the score. Signed credentials let another merchant verify selected attributes, such as a purchase threshold or trust band, without needing the customer's full transaction history.

For testing at scale and across multiple interacting players, we built a deterministic shopping economy in SpacetimeDB, with 10,000 simulated shoppers and 100 simulated merchants. Shoppers purchase, receive items, request refunds, and accumulate histories. The globe, event stream, merchant cashflow view, and agent timelines expose those events and their consequences, demonstrating how each shopper interacts with multiple merchants and vice versa. Behavioral detectors flag coordinated claims, repeated defect patterns, and unusual refund frequency for CREAO's Pattern Analyst to review.

For finance teams, the Refund Finance Desk exposes backend simulation analysis of cash, accounting, conditional obligations, reserves, and close. SpacetimeDB stores finance events; deterministic code reconciles decision, processor, bank, warehouse, and ledger records. CREAO provides impact, review, close, forecast, and exposure roles, with outputs validated before use. A synthetic one-million-order book supplies planted errors and legitimate decoys.

How we built it

We built Vouch around three essential systems: Factory Droid SDK for tool-using video investigation, CREAO for specialized review and trust judgments, and SpacetimeDB for the shared economy, operational records, and enforceable transaction rules. A Next.js interface and Node.js worker connect those systems into one customer-to-merchant workflow.

CREAO Agents

Vouch's core refund decisions run on CREAO agents.

Vouch Refund Investigator (core decision agent)

Recommends accept, decline, or more_evidence for a single refund claim, with amount, method, cited reasons, and confidence.

Vouch Risk Scorecard

Reads every contextual indicator for a refund claim and returns a JSON scorecard of supports, neutral, and caution readings. It never recommends a decision.

SpacetimeDB: shared state and enforceable rules. SpacetimeDB holds shoppers, purchases, claims, analysis steps, artifacts, findings, recommendations, decisions, and score changes. Its scheduled reducers run the simulated economy inside the database, while browser subscriptions keep the customer's progress tracker, merchant case, globe, and trust history synchronized. Authorization, controller escalation, and duplicate-update checks live in SpacetimeDB reducers. Without SpacetimeDB, Vouch loses its shared case state, running economy, and consistent record of trust decisions; it is the operational backbone.

Preparing evidence for Factory Droid. The worker uses FFmpeg and ffprobe to inspect and normalize the upload, extract audio, sample frames, and create timestamped contact sheets; faster-whisper supplies word-level transcription. Factory Droid's image checks inspect the randomized finger challenge, while deterministic probes examine reused footage, blank or frozen frames, scene changes, and metadata inconsistencies. Each result becomes evidence the investigator can inspect, with limitations recorded when a tool cannot complete its work. These checks establish internal consistency, not proof of authenticity.

Factory Droid SDK: a custom investigation loop. We built a case workflow on top of the SDK, with custom MCP tools including get_case, get_policy, get_customer_summary, list_artifacts, list_findings, and recommend_refund. Factory Droid's Case Analyst retrieves the customer issue, order, policy, history summary, and video evidence in one consistent case format. It works across the timeline, revisits frames, runs measurements or writes probes, inspects its own findings, and records evidence for and against each statement. Every case follows the same report schema while preserving its specific evidence. Tool calls, findings, files, and fence decisions form a run record that merchants can inspect. The permission handler checks consequential operations, including nested script calls, without giving the agent payment authority.

Factory Droid Report Auditor and merchant verification. Each finding must cite a registered artifact and a time window. The worker builds proof clips and audits artifact existence, hashes, timing, and coverage. An additional Factory Droid Report Auditor can independently test findings against those artifacts. Rejected findings do not contribute to the evidence score. Merchants receive the same timestamped findings, proof clips, method, and limitations, so they can verify the investigation themselves rather than accept an unexplained model verdict.

CREAO: the trust-decision layer. Saved CREAO agents occupy distinct roles for evidence analysis, refund investigation, policy and risk assessment, pattern review, and Vouch Score judgment. The worker calls them through CREAO API Trigger, polls or receives a webhook, validates their outputs, and records each run's provenance. The live Refund Investigator and Risk Scorecard panel judge how a reviewed claim should affect the shopper's Vouch Score; SpacetimeDB applies the bounded change once. The configured Refund Reviewer role is also required before eligible automatic approvals. CREAO is essential to this agent-reviewed trust workflow: disabling it stops CREAO-dependent automatic approvals and fraud confirmations instead of silently replacing them with unreviewed decisions. The economy and evidence preparation can continue, but those judgment paths cannot complete as designed. Missing review roles leave cases pending or with a person.

CREAO judgments and separate scoring rules. A transparent, hand-set formula combines Factory Droid's audited findings, claim coverage, integrity checks, and historical context to choose a refund route. The longer-term Vouch Score is a separate record updated through CREAO judgments and bounded SpacetimeDB reducers. Keeping the two distinct lets a merchant inspect the current claim rather than treating reputation as proof that every request is valid. The evidence formula is a prototype heuristic, not a calibrated fraud probability.

Portable credentials backed by SpacetimeDB. We used Ed25519 signatures, salted SHA-256 digests, and nonce-bound presentations. Verification checks the issuer's signature, expiration and revocation, disclosed attributes, holder proof, and the receiving merchant's policy. SpacetimeDB stores the credential registry and revocation state; blockchain deployment remains future work.

SpacetimeDB controls and CREAO finance roles. SpacetimeDB separates proposing, deciding, and posting. A separate poster identity records balanced entries, and refunds above $500 require controller approval. The backend finance extension adds integer-cent accounting, period locks, conditional return obligations, and a deterministic reconciliation verifier. CREAO's finance roles interpret impact, close, forecast, and exposure results through validated outputs; arithmetic and ledger authority remain in code. Tests cover these controls, seeded simulation behavior, evidence handling, credential tampering, and agent-output validation.

We also implemented an experimental Factory Droid Detector Lab in which Droid can propose detector changes inside an isolated candidate folder, run a fixed evaluator, and expose its diff and results through SpacetimeDB records for human promotion.

CRS: identity and contact-risk screening, being integrated. The existing backend adapter uses CRS's Fraud Finder and TransUnion TruValidate services to normalize contact-risk signals and identity matches or discrepancies. In the planned claim flow, those results will give CREAO's investigator and risk agents context about the submitting account alongside its Vouch Score. Cases can continue to refund investigation, request additional verification, or go to merchant identity review. A mismatch or unavailable result calls for more verification, not an automatic refund rejection. Identity screening remains separate from the question of whether the video supports the refund; wiring it into ordinary claims is still in progress.

Challenges we ran into

Giving agents useful tools within enforceable limits. Droid's tool behavior differed between autonomy settings, and script batches required inspecting their nested calls. We built explicit permission checks and recovery messages so a refused operation did not silently end an otherwise useful investigation.

Handling asynchronous and imperfect agent outputs. CREAO runs can take minutes, and live responses do not always match the expected structure. We separated background execution from HTTP requests, showed progress through database rows, and added defensive parsing, validation, retry limits, and labeled fallbacks.

Keeping the shared simulation responsive. The worker needs detailed history, while a browser needs a manageable view. We used indexed worker-side maps, aggregate merchant-flow records, bounded event feeds, throttled rendering, and subscriptions scoped to the selected case or shopper.

Following a refund through time. An approved refund, money settled, an item returned, and an obligation accrued can occur in different periods. The finance extension models those as separate events and specifies which source answers each question. Its remaining integration work is explicitly tracked.

Accomplishments that we're proud of

  • An inspectable refund workflow: video evidence becomes findings, proof clips, a reviewed recommendation, and a recorded outcome.
  • Agents doing product work: Factory investigates cases with tools; CREAO contributes review, pattern analysis, and trust judgments through defined roles.
  • A reproducible environment for testing trust: synthetic shoppers generate histories and adversarial behavior, with hidden ground truth available for evaluation.
  • Controls behind the interface: database rules enforce approval limits, role separation, balanced entries, and one score review per refund.
  • A foundation beyond support tickets: signed credentials and the finance extension connect one customer's claim to merchant trust and operational consequences.

What we learned

We learned that trust has to be built into the full decision process: what an agent can read, what evidence it cites, which actions it can propose, who can approve them, and how an error can be traced afterward.

We also learned that claim validity, customer history, and payment status are different questions. Combining them into one confidence number hides the information a merchant needs. A credible history can justify a faster route, but it cannot replace evidence or store policy.

Finally, simulation gave us repeatable histories and difficult cases within a weekend, while showing us what remains unproven. Synthetic detector results do not establish real-world fraud accuracy, and a local simulation benchmark does not establish production capacity. Those require merchant pilots and measurements on actual workflows.

What's next for Vouch

1. Move from a simulated economy to real commerce. We will connect merchant order systems, shipping and warehouse events, and payment processors so Vouch can investigate actual purchases and execute approved refunds with settlement tracking. We will pilot with merchants, measure review time, false flags, customer outcomes, and investigation cost, and use reviewed outcomes to calibrate the prototype's scoring rules. Scaling beyond 10,000 shoppers will require efficient subscriptions, durable job scheduling, and investigation workers that can process many independent cases concurrently.

2. Expand beyond refunds into agent-driven financial workflows. The reusable process is evidence, policy, authority, action, and a verifiable outcome. We want to apply it to shopping-agent purchases, exchanges, warranties, subscription disputes, reimbursements, invoice disputes, and payment reconciliation. The finance extension is our first step toward connecting those actions to cash planning, reserves, accounting, and close. Each workflow will have its own evidence requirements and approval boundaries.

3. Become the "credit score" for the agentic commerce era. Our ambition is a portable, verifiable record of transactional reliability that businesses can use when a person or an authorized agent acts across services. Companies need validity and accountability; consumers need speed and ease. Vouch should let earned trust carry between merchants without making every customer start from zero at every store.

4. Earn trust as the custodian of consumer data. Accurate records and privacy must develop together. Merchants should receive the evidence for their own claims and the verified attributes needed to apply their policies, without access to a consumer's detailed purchases at other stores, where they shopped, or when. We will strengthen selective disclosure, merchant-specific permissions, access logs, retention limits, and ways for consumers to correct disputed records. As the entity collecting valuable shopping data, Vouch must be accountable to consumers as well as merchants. We intend to charge for verification and workflow services, with an explicit commitment against brokering or selling individual purchase histories.

Built With

Share this project:

Updates

Submission history