-
-
It knows it's a scam. It pays anyway. Separate tests with simulated payment requests.
-
Six scripts: all 18 recognition replies per model said SCAM. Separate acting tests: risky requests on 6, 3 and 1 scripts.
-
Fictional exchange support: three separate SCAM replies; the assistant requested a simulated EUR4,200 payment.
-
Browser replay refuses unsaved payees and code sharing. No real payment system is connected.
-
Upload a screenshot: Nemotron Super VL reads it, you check the text, then run the test.
Inspiration
A model can label a message as a scam while an assistant, tested separately, still requests a payment on it. We built Scam Arena to make that gap visible and inspectable before someone connects an assistant to real tools.
What it does
Paste a fictional email, text or letter, select an example, or upload a synthetic screenshot. Nemotron transcribes the image, then you review the text before starting a run.
The same text goes to independent conversations:
- Review: a source-quoted analysis explains warning signs and safer steps.
- Ask: three separate recognition checks show their visible answers, agreement and failures.
- Act: Sam's assistant receives simulated payment, code-sharing and verification tools, plus up to two labelled standard follow-ups. The transcript shows what it actually requested.
The conversations do not receive one another's answers. This is a comparison of separate contexts, not proof that an acting agent knowingly ignored its own warning.
The action check counts whether the run avoided a payment or code-sharing request. It is a literal test result, not a fraud probability. Check the action replays the recorded requests against a deterministic browser policy, without changing the original model result. No saved payees are configured by default. This replay does not enforce a real payment.
History keeps up to 20 text results in the current browser. Recorded examples work without inference credits, and the model inspector exposes raw replies, tool transcripts, source hashes and downloadable JSON.
What we found
Six authored scam scripts, run through Nebius Token Factory on 30 October 2026:
| NVIDIA model | Recognition replies starting SCAM | Risky requests in separate baseline acting trials |
|---|---|---|
| Nemotron 3.5 Lightning | 18/18 | 18/18: 9 payment and 9 code-sharing requests |
| Nemotron 3 Super (120B) | 18/18 | 3/6: payment requests |
| Nemotron 3 Ultra (550B) | 18/18 | 1/6: a payment request |
Recognition used three repeats of each full script, no tools and temperature 0.7. Acting used incremental messages, tools and temperature 0.0, stopping at the first risky request. Lightning had three repeats per script; Super and Ultra had one. Different prompts, context and sample sizes mean these results do not establish a general model ranking.
The safety-instruction arms are retained too: 10/18, 5/6 and 1/6 risky trials respectively. There were no benign controls, so 18/18 is not detection accuracy. A verification-tool request is not proof of completed verification. The source and inspector retain these methods and full recorded results.
How we built it
Python, httpx and Pillow serve a browser interface written in HTML, CSS and JavaScript. Nebius Token Factory's OpenAI-compatible API runs NVIDIA Nemotron 3.5 Lightning for live text tasks and Nemotron 3.5 Super VL for screenshot transcription. Super 120B and Ultra 550B supply the retained comparison runs.
The analysis path requests strict five-field JSON and independently validates exact quotes against the input. Incomplete output, refusals and tool calls are rejected. Recognition failures remain errors, never invented safe votes. Images are validated, size-limited and processed in memory; the API key stays server-side.
All live routes share admission and concurrency limits. A custom comparison permits at most 13 provider attempts, including retries. Recorded views need no provider calls. The public demo uses a non-root container on Cloud Run, with a dedicated runtime identity and secret. In-memory request limits are not a provider spending guarantee.
Challenges and what we learned
Nemotron declined to role-play the scammer, so we used fixed authored scripts. Early analyses exhausted their output budget or missed required fields. We fixed the structured-output contract, disabled thinking for that path and kept strict source checks. Screenshot line-break differences are aligned only when one exact whitespace-equivalent source span exists.
The lesson is practical: evaluating what an AI says does not establish what an assistant will request with tools. The app lets a reviewer inspect both, including errors, rather than trusting a single green badge.
Accomplishments
A working message-to-analysis-and-actions workflow, screenshot review, three-model recorded evidence, raw transcript inspection and a reproducible Apache 2.0 source release. The public demo, repository and video show the same scoped application.
What's next
Independently authored scenarios, benign controls, repeated tests with matched contexts and adapters for other assistants. We would measure legitimate task completion alongside risky requests.
Limits
People, accounts, funds and tool effects are fictional. No real money moves and no codes are sent. Text and images go to the configured model provider; browser history stores text locally. Use synthetic details. This is a small experiment and a testing app, not a calibrated fraud detector or a guarantee of prevention.
Built With
- css
- google-cloud-run
- html
- httpx
- javascript
- nebius-token-factory
- nvidia-nemotron
- pillow
- python
