Inspiration

Our inspiration came from the x402 protocol in the Web3 world. x402 revives the HTTP 402 "Payment Required" status code so software, including AI agents, can pay for things on its own as part of a normal web request. It showed us that agents paying autonomously is a real, practical idea.

That made us ask: what does this look like in Web2? Most people don't pay with crypto wallets. They pay with PayPal, cards, and online stores. For agentic commerce to reach everyday users, an agent has to shop and pay through the systems people already use, and people have to feel safe giving it that power.

So we built Penny, an agent for Web2 commerce on top of PayPal, where the user decides what payment flow they desire: pay yourself, or let Penny pay with a saved payment method you can revoke at any time.

What it does

Penny is an AI shopping agent. You tell her what you want with text or a photo, she searches real products, shows you options, and helps you pay through PayPal.

She has two modes:

  • Shop with me: Penny finds products and creates the order. You approve the payment yourself on the PayPal login screen.
  • Buy for me: You save your PayPal payment method once, and Penny completes checkout on her own. You can delete that saved method at any time, which turns the mode off and returns you to Shop with me.

How we built it

Penny is a multi-agent workflow built with LangGraph. Every message goes to a coordinator agent that classifies the user's intent and routes it to the right agent:

  • Text product search and image product search, both powered by Channel3
  • System enquiries, where users ask what Penny can do
  • Order creation and payment capture through the PayPal Orders API. The key design choice is how the two modes work. After items are gathered, an auto payment decision step checks whether the user has a saved PayPal vault token:

Let \(T = 1\) if the user has a saved vault token, and \(T = 0\) if not:

$$ T = 1 \;\Rightarrow\; \text{auto capture (Buy for me)} $$

$$ T = 0 \;\Rightarrow\; \text{manual approval on PayPal (Shop with me)} $$

Authenticated FastAPI endpoints let each user create a vault setup token, exchange it for a vault payment token, and delete it whenever they want.

Stack: Python, TypeScript, React.js, FastAPI, LangGraph, DeepSeek, Channel3, PayPal Orders and Vault APIs (sandbox).

Challenges we ran into

  • Intent misclassification. When the classifier was unsure, our routing function sent unknown intents into the order flow. We changed unmatched intents to fall back to a safe path instead of starting a purchase.
  • Images as bytes, not strings. Passing raw image bytes through a tool declared as a string caused confusing failures, so we normalized images before they reached the search tool.

What we learned

  • Let the language model propose and let deterministic code decide, especially when money is involved.
  • Trust is a product feature. Letting users opt in, and revoke access in one step, mattered more than any clever prompt.

Built With

Share this project:

Updates

Submission history