AgenticAudit

Making AI-powered commerce trustworthy through cryptographic proof, transparent transactions, and intelligent dispute investigation.

Comment

AgentAudit — Trust, Transparency, and Accountability for AI Shopping Agents

Inspiration

AI agents are rapidly evolving from tools that answer questions into systems that can make decisions and perform actions on behalf of people. Imagine asking an AI agent to find the best headphones within your budget. It compares products, selects one, creates an order, and initiates a payment.

But what happens when something goes wrong?

What if the agent purchases the wrong product? What if a customer disputes a transaction? How can a business, payment provider, or customer determine what actually happened?

We were inspired by this emerging gap between AI autonomy and financial accountability.

Traditional payment systems record transactions, but they do not necessarily provide a complete, understandable history of the decisions made by an AI shopping agent before a purchase.

We wanted to build a system that makes AI-driven commerce more transparent, verifiable, and accountable.

That idea became AgentAudit.

What it does

AgentAudit is an AI-powered auditing and dispute intelligence platform designed for AI-driven commerce, with PayPal integration.

It focuses on three major capabilities:

1. Digital Notary — Cryptographic Audit Trail

AgentAudit records observable AI shopping agent actions in an append-oriented, SHA-256 hash-linked ledger.

Each recorded event is connected to the previous event, creating a verifiable sequence of decisions and actions.

The system records events such as:

  • Customer shopping requests.
  • Extracted product requirements.
  • Product comparisons and rejected options.
  • Final product selection.
  • Explicit customer purchase authorization.
  • PayPal order creation and payment capture.

The ledger can be verified to detect modifications to recorded events.

2. Automated Dispute Desk

AgentAudit connects PayPal dispute events with the corresponding shopping sessions.

Its dispute intelligence pipeline:

  • Receives PayPal dispute webhooks.
  • Verifies webhook signatures.
  • Stores webhook events with idempotency protection.
  • Correlates disputes with purchase sessions using transaction identifiers.
  • Verifies the cryptographic audit trail.
  • Generates structured evidence briefs.
  • Identifies missing evidence and inconsistencies.

The system is designed to distinguish between recorded agent actions, customer authorization, payment records, and fulfillment evidence rather than treating them as interchangeable.

3. AI Dispute Investigator

The AI Dispute Investigator analyzes structured dispute evidence and produces an understandable investigation report.

It identifies:

  • Important events in the transaction timeline.
  • Evidence supporting the transaction.
  • Evidence contradicting the transaction.
  • Potential inconsistencies.
  • Missing information.
  • Recommended next steps.

It integrates with Google's Gemini API and includes a deterministic offline fallback so that basic investigation functionality remains available when AI services are unavailable.

The AI assists with evidence analysis; it does not independently decide legal responsibility or fabricate missing evidence.

How we built it

We built AgentAudit using a full-stack architecture.

Backend

  • Rust
  • Axum
  • Tokio
  • SQLx
  • SQLite for local development
  • SHA-256 cryptographic hashing
  • RFC 8785 JSON canonicalization
  • PayPal Orders API
  • PayPal Disputes API and webhook integration
  • Google Gemini API integration

Frontend

  • React
  • TypeScript
  • Vite
  • Custom operations dashboard

AI and intelligence

  • Structured AI investigation reports.
  • Evidence-grounded prompts.
  • Evidence reference validation.
  • Sensitive-data redaction.
  • Deterministic fallback investigation engine.

The backend manages the shopping workflow, payment authorization, cryptographic ledger, webhook processing, dispute correlation, and AI investigation.

The frontend provides an operations dashboard for inspecting shopping sessions, reviewing transaction timelines, verifying ledger integrity, and investigating disputes.

We also implemented explicit customer purchase authorization and maximum authorized spending limits. Product selection alone is not treated as permission to capture a payment.

Challenges we faced

1. Building a verifiable audit trail

One of our biggest challenges was ensuring that recorded agent events could be verified consistently.

We implemented a hash-chain design using SHA-256 and canonical JSON serialization. This required careful handling of event ordering, payload encoding, transaction boundaries, and hash verification.

We also learned an important distinction: a hash chain can detect modifications relative to a trusted chain, but a database owner who can rewrite the entire database and recompute every hash could still manipulate it. Stronger guarantees require external anchoring or independent signatures.

2. Handling real payment workflows

Integrating PayPal required us to distinguish between creating an order, receiving customer approval, and successfully capturing a payment.

These are different stages, and treating them as the same would create misleading audit records.

We implemented PayPal Sandbox order creation, approval, and capture flows, while recording the corresponding events in the audit ledger.

3. Reliable webhook processing

Dispute webhooks introduced additional complexity.

We needed to preserve raw request payloads, verify PayPal signatures, prevent duplicate event processing, and handle retries safely.

We also implemented worker lease fencing to reduce the risk of multiple workers processing the same event concurrently.

4. Correlating disputes with purchases

A dispute must be connected to the correct shopping session.

Matching transactions using only customer names, email addresses, amounts, or timestamps would be unreliable.

We implemented a correlation hierarchy based on transaction identifiers, with uncertain matches routed for manual review rather than inventing a relationship.

5. Making AI evidence-grounded

An AI-generated dispute report is only useful if its claims can be traced to actual evidence.

We therefore designed the investigator around structured evidence, reference validation, missing-evidence reporting, and a deterministic fallback.

The goal was not simply to add an AI chatbot, but to create an investigation assistant that explains what the available evidence supports and what remains unknown.

6. Deployment and infrastructure

Preparing the application for public deployment also introduced challenges involving environment configuration, Rust build compatibility, database persistence, and frontend-backend connectivity.

We learned that deploying a frontend does not automatically deploy its backend, and that temporary cloud storage is not suitable for preserving an audit ledger.

These challenges helped us think beyond getting a demo running and consider the operational requirements of a trustworthy financial application.

What we learned

Building AgentAudit taught us that AI autonomy must be accompanied by accountability.

Some of our biggest takeaways were:

  • AI actions need observable and verifiable records.
  • Customer authorization must be distinct from agent recommendations.
  • Payment order creation and successful capture are separate events.
  • Cryptographic integrity does not automatically establish the truth of an event.
  • Webhook processing requires strong idempotency and transaction safety.
  • AI-generated conclusions should always be grounded in traceable evidence.
  • Missing evidence should be reported honestly rather than filled with assumptions.
  • Financial automation needs clear boundaries, human oversight, and reliable auditability.

Most importantly, we learned that trust in AI commerce is not just about making agents smarter. It is about making their actions understandable and accountable.

What's next

We envision AgentAudit becoming a trust and accountability layer for the growing ecosystem of autonomous commerce agents.

Future improvements include:

  • Persistent PostgreSQL-backed production storage.
  • External cryptographic anchoring for stronger tamper evidence.
  • More complete PayPal dispute evidence submission workflows.
  • Additional payment provider integrations.
  • Expanded AI-powered dispute response assistance.
  • AI agent behavior auditing and anomaly detection.
  • Stronger customer authorization and transaction policy controls.

Built for the future of accountable AI commerce

AgentAudit is our attempt to answer an important question:

When an AI agent acts on someone's behalf, can we prove what it did, why the transaction proceeded, and what evidence supports the outcome?

We believe transparent, verifiable records are an essential foundation for trustworthy autonomous commerce.

That's what we're building with AgentAudit.

Built With

Share this project:

Updates

Submission history