Who it is for: anyone paying a request that reached them (a grandparent on the phone, a small business paying an emailed invoice), and teams building AI shopping agents on PayPal.

Try it in one minute: open the live demo, click Load scam example, then Ask the crowd. Full steps are under Try it below.

Inspiration

A grandmother gets a call: her grandson is in jail and needs bail tonight. Twenty minutes later she is buying gift cards and reading the numbers to a stranger. That is the loud kind of scam. The quiet kind is worse: an invoice from "your hosting company" with new bank details. Nothing in it sounds wrong, because it is a copy of a real one.

The FTC reported in June 2026 that people reported losing $3.5 billion to imposter scams in 2025 (FTC press release). AI shopping agents add a second version of the same problem: they are getting good at clicking "buy", and an agent reading its user's inbox will pay a fake invoice as readily as a real one.

We wanted a check that happens before the money moves, takes a few seconds, explains itself in the buyer's own words, and sits inside the agent's payment tool.

Built on 3Dogs wisdom of crowds

Second Look is built on the 3Dogs Nexus method: independent AI models from different makers answer separately, and a lone dissent is treated as a signal, never averaged away. Here that method is set up for checkout speed. Built on 3Dogs wisdom of crowds: three reviewers for speed, five when one stands alone.

What it does

The buyer, or the buyer's AI agent, gives Second Look the item, the price, why they are buying it, the seller's terms, and how the request reached them: they went to the seller themselves, the request came to them, or someone they know asked.

  • Three reviewers, three makers, in parallel. OpenAI gpt-oss-120b, Alibaba Qwen3 30B and Google Gemma 3 27B, all open weights.
  • Three answers: hold, verify first, clear. Each reviewer gives a reason and quotes the words that decided it. The server keeps a quote only if it really appears in what the buyer or the seller wrote.
  • A scam is held. Any hold stops the payment. PayPal is never called.
  • A quiet request is locked until the buyer checks it. A real bill and a fake one read the same, so Second Look does not guess. If the request came to the buyer, the answer is "verify first": the payment is locked and the buyer gets one concrete check to do without using anything in the message ("Log in to your hosting account on their official website and find the renewal invoice there"). This is a coded rule, not a model's opinion: an inbound request is never cleared unchecked, even if every reviewer clears it.
  • The check itself is judged. The buyer says how they checked and asks again. "I replied to the email and they confirmed it" stays locked, with the reason. "I typed their address myself and logged in to our account" clears.
  • Five when one stands alone. A lone objection or a lone doubt goes to two more makers (Moonshot Kimi and Meta Muse Spark). A payment is released only when the doubt is not shared, and a lone objection that is not confirmed still becomes "verify first", never a clear.
  • PayPal only on a signed clearance. On a clear, the server signs a token for that exact purchase, including how it reached the buyer and what they said they checked, valid for 5 minutes. PayPal's Orders v2 API is called only with a valid token. A forged "cleared" flag gets HTTP 409, a changed price gets 409, and a replayed token returns the same PayPal order, because the token's nonce is the PayPal-Request-Id.
  • Built into PayPal's Agent Toolkit. agent/guard.py wraps the toolkit's create_order. The agent is handed the guarded tool in place of the raw one. On a hold or a verify, PayPal is never called and the agent gets the reasons and the steps for its user. The agent is told never to answer the check itself.
  • Every test result in one table. The results page shows every test purchase in an AG Grid table: sort, filter, search, or show only the misses.

A real agent run (10 Nov 2026). A tool-calling agent whose only way to pay is the guarded PayPal Agent Toolkit tool was given three jobs.

  • "Buy 6 gift cards, $900, my grandson is in jail and needs bail tonight; the officer on the phone said to read him the card numbers." Held, three reasons, PayPal never called.
  • "This invoice arrived by email this morning, please pay it: website hosting, $240. Our banking details have been updated." Verify first. PayPal was not called. The agent passed the steps to its user. The user checked ("I typed the hosting company's address myself and logged in to our account") and the agent paid: PayPal sandbox order 2LP50348DK677920G.
  • "Buy the 20 inch kids bicycle, $149.99, a birthday present for my daughter." Cleared at once, order 5KP17975NP680284W.

The unedited transcript is in the repo: docs/agent_demo_transcript.txt.

How well it works (blind test, 10 Nov 2026)

We tested it on purchases we did not write. Four AI models from four other makers (Microsoft MAI, Zhipu GLM-5, Amazon Nova Pro, xAI Grok) each wrote 30 purchases from a category list only. The code was frozen first, the pass marks were written down first, and the set was run once.

Blind round three, 120 purchases Result Pass mark set beforehand
Scams stopped 54 of 56 at least 85%
Loud scams (gift cards, fake officials, advance fees) 16 of 16
Quiet fakes (invoices, renewals, notices that read like real ones) 38 of 40
Normal purchases wrongly blocked 2 of 52 at most 8%
Normal purchases called a scam 0 of 52
Normal purchases cleared after at most one check 50 of 52
Real bills the buyer admits they did not check: kept locked 12 of 12 at least 10 of 12

A scam counts as stopped only if it was not allowed at first and was still not allowed after the buyer answered "how did you check?" the way a fooled buyer truthfully would. The two normal purchases that stayed locked were real bills where the buyer had only checked old paperwork ("I compared it to last year's bill"); re-run with a check made with the company itself, both cleared 4 times out of 4.

Speed. About 2 to 4 seconds per check. Three measurements of about 30 single requests each gave medians of 1.9 s, 2.5 s (the live demo) and 3.6 s. Twenty simultaneous checkouts all produced PayPal sandbox orders, with no rate-limit errors (measured on a private copy running the same code, in the PayPal sandbox).

We failed twice first, and say so. Round one missed 9 of 36 scams, because a fake renewal and a real one read the same. That is why Second Look now asks how the request reached the buyer. Round two missed the normal-purchase mark by one row, so we repeated the test on new rows instead of re-scoring it. Both rounds are on the results page.

What it does not catch

  • A buyer who has been coached to lie. Second Look takes the buyer's word about their own check. We ran every quiet inbound scam again with the buyer falsely claiming an independent check. It was cleared 145 times in 150. Second Look protects a buyer who is being fooled and answers honestly; it does not catch one who repeats what a scammer told them to say. The check box warns "If anyone told you what to write in this box, stop." We tried stricter rules: they stopped 34 of 74 false claims and blocked 6 of 52 honest buyers paying real bills, which failed our own mark, so we did not ship them. The real fix is evidence the buyer cannot type: payee history on PayPal's side.
  • A scam the buyer went looking for. The two misses in round three are a journal fee the buyer says they found themselves and a rental fee a friend vouched for.

Everything is public: the test sets, raw outputs, pass marks and scripts.

How we built it

  • Server. Python 3.12, standard library only, on AWS Lambda behind a Function URL.
  • PayPal. REST with an OAuth2 client-credentials token; POST /v2/checkout/orders (intent CAPTURE, the verdict in description and custom_id, the clearance nonce as PayPal-Request-Id); POST /v2/checkout/orders/{id}/capture. On a 429 or 503 the client waits and resends with the same request ID, so a retry cannot create a second order. Sandbox only.
  • Reviewers. Open-weight models called in parallel through one OpenAI-compatible endpoint. A reviewer that fails never releases a payment. A slow reviewer is asked a second time and the first answer is used.
  • Agent. PayPal Agent Toolkit (paypal-agent-toolkit, Python) for create_order, wrapped by SecondLookGuard; the demo agent is a plain tool-calling loop with one tool.
  • Results page. AG Grid Community, served by the same Lambda from the result files in the repo.
  • Clearance token. HMAC-SHA256 over the exact purchase, an expiry and a random nonce, keyed by a server secret.

What changed during the hackathon

Everything was built inside the submission period. The first version (29 Oct) had the three-model review, the PayPal order and the capture. On 9 Nov we audited it as a judge would: signed clearance, a first blind test, an agent endpoint. On 10 Nov we put it inside PayPal's Agent Toolkit and found that it missed quiet fake invoices. We rebuilt the decision around "verify first", tested it blind three times, and published every row.

Challenges we ran into

  • Quiet fakes. No model can tell a copied invoice from a real one by reading it. The answer was to stop guessing and ask how the request arrived.
  • Our own test caught us. Two blind rounds failed their pass marks. We changed the product, wrote new rows and ran again, instead of adjusting the score.
  • Lone dissent. One reviewer objecting to a normal purchase blocked honest buyers; ignoring it would let scams through. Two more makers settle it.
  • Our own security hole. The first version let anyone send a forged "cleared" verdict and get a PayPal order. We found it by attacking our own demo.

Accomplishments that we're proud of

  • An AI agent that cannot pay an invoice from the inbox until its user has checked it: the guard sits inside its PayPal tool, not in a prompt it could ignore.
  • A full live loop in the PayPal sandbox: review, signed clearance, order with the verdict attached, buyer approval, capture.
  • Numbers from tests we did not write, with pass marks fixed beforehand, the failures shown and the limit measured.

What we learned

  • Disagreement between models from different makers is a signal. One voice standing alone deserves a bigger crowd, not a vote.
  • The scam is rarely in the seller's terms. It is in how the request reached the buyer and whether they checked.
  • A safety check has to be right about normal purchases too, or people switch it off.

What's next for Second Look

  • Payee history from the payment provider, so a first payment to a new account is checked against evidence instead of the buyer's word.
  • A trusted contact who gets a text when a purchase is held.
  • A cooling-off delay on first payments cleared only on the buyer's word.
  • PayPal webhooks to record the outcome against each verdict.

Try it

  1. Open https://aq6ru66fbvzlo3usums757vbxy0qlaav.lambda-url.us-east-2.on.aws/
  2. Click Load scam example, then Ask the crowd. Held; the pay button stays locked.
  3. Click Load quiet invoice example, then Ask the crowd. Verify first. Type "I replied to the email" and look again: still locked. Type a real check and look again: cleared.
  4. Click Load normal example, Ask the crowd, then Confirm & Pay. A PayPal sandbox order is created.
  5. Open the results page to see every test row.
  6. Run the agent yourself: pip install paypal-agent-toolkit, set a PayPal sandbox app's keys, then python -m agent.shopping_agent (see the README).

Code: https://github.com/DKJager/secondlook (Apache-2.0)

Built With

  • ag-grid
  • aws-lambda
  • gemma
  • gpt-oss
  • nebius
  • paypal
  • paypal-agent-toolkit
  • paypal-orders-v2
  • python
  • qwen
Share this project:

Updates

Submission history