AgentGuard

A security firewall that stops AI agents from making unauthorized payments.

Comment

Inspiration

AI agents are becoming capable of making purchases and managing tasks with very little human involvement. While this makes automation useful, it also creates a serious problem: an agent could misunderstand instructions, get manipulated by a malicious invoice, or spend more money than its owner intended.

We wanted to build a safety layer between AI agents and payment systems.

That idea led to AgentGuard. Our goal was to make autonomous payments safer without removing the convenience of automation. We believe AI agents should be able to make purchasing decisions, but they should never have unlimited financial authority.

What it does

AgentGuard is a security layer that monitors payment requests made by AI agents before they reach PayPal.

It checks every purchase against spending rules, verifies merchants, scans invoice details for suspicious instructions, and calculates a risk score.

Based on these checks, it takes one of three actions:

  • Approve: The transaction passes the configured security checks.
  • Review: The transaction needs human attention before settlement.
  • Block: The transaction violates a security policy and is stopped before reaching PayPal.

AgentGuard uses PayPal's authorization and capture flow to hold authorized funds until the required human approval is received. A reviewer can approve the payment or reject it and release the authorization.

It also provides a dashboard where users can review transactions, inspect security alerts, manage spending policies and monitor payment activity.

How we built it

We built AgentGuard using Python, FastAPI and SQLite for the backend, and React, TypeScript and Tailwind CSS for the frontend.

The system has several main components:

  • MCP Gateway: Receives purchase requests from AI agents and ensures they pass through AgentGuard.
  • Policy Engine: Enforces spending limits, daily budgets and merchant allowlists using deterministic rules.
  • Invoice Scanner: Looks for prompt injection, suspicious payment instructions, invoice calculation errors and merchant mismatches.
  • Risk Engine: Combines security signals into a risk score to help decide whether a transaction should proceed.
  • PayPal Integration: Uses PayPal Orders v2 and Payments v2 to support authorization, capture and void operations.
  • Approval Dashboard: Allows a human reviewer to inspect flagged transactions and control their final settlement.

We also built automated tests and attack simulations to check how the system responds to suspicious and normal purchase requests.

Challenges we ran into

One of our biggest challenges was designing a payment flow that separates authorization from capture. Creating an order is not the same as holding funds, and capturing a payment must only happen after the required checks and approvals.

Another challenge was detecting prompt injection in invoice content. Invoices are not always trustworthy. They can contain instructions that try to manipulate an AI agent into ignoring its original rules. We had to make sure these instructions were treated as untrusted data.

We also faced challenges with transaction states, duplicate requests, idempotency and concurrent payment operations. A repeated request should never result in an unintended duplicate payment.

Finally, connecting the backend, MCP gateway, PayPal integration, and frontend into a single consistent workflow required careful testing.

Accomplishments that we're proud of

We are proud to have built a complete, security-focused payment workflow rather than just another AI chatbot.

Some of the things we are particularly proud of:

  • Building a gateway that places security checks between an AI agent and payment execution.
  • Implementing deterministic spending controls that an LLM cannot override.
  • Adding invoice security checks for prompt injection and suspicious payment information.
  • Designing a three-way decision system: Approve, Review or Block.
  • Building a human approval workflow around PayPal authorization and capture.
  • Creating a dashboard to inspect transactions, risks and approval requests.
  • Developing automated security scenarios to test both suspicious and normal purchases.

Most importantly, we focused on ensuring that an AI agent does not get unlimited control over someone else's money.

What we learned

We learned that building AI-powered financial applications requires more than connecting an LLM to a payment API.

Security decisions involving money should not depend entirely on an AI model. Deterministic rules, proper authentication, transaction state management and clear approval boundaries are equally important.

We also learned how MCP can connect AI agents with external tools, how PayPal's authorization and capture lifecycle works, and why idempotency and concurrency matter in payment systems.

Another important lesson was that a good security system must explain why it blocked or flagged a transaction, rather than simply returning an error.

What's next for AgentGuard

We want to continue improving AgentGuard beyond the hackathon prototype.

Our next priorities are:

  • Improve invoice analysis and detection of more advanced prompt injection attacks.
  • Add stronger merchant identity and payment destination verification.
  • Improve authentication and role-based access for human reviewers.
  • Add PayPal webhooks for better payment status synchronization.
  • Strengthen transaction handling for concurrent requests and multiple application instances.
  • Expand security testing with larger and more diverse attack datasets.
  • Improve the dashboard with clearer transaction explanations and security analytics.
  • Explore support for additional payment providers and AI agent frameworks.

Our long-term goal is to make AgentGuard a reusable security layer that enables AI agents to make payments while maintaining human financial authority.

Built With

Share this project:

Updates

Submission history