Injection-Resistant Inbox Agent

Inspiration

AI email assistants can read, forward, reply to, and delete emails. But an attacker can hide commands inside an email, and the assistant may follow them. This is called prompt injection. We wanted to see how bad it is on a real model, and how to stop it.

What it does

It is a defense system that sits in front of an AI email agent. It blocks attacks before the agent can forward, reply to, or delete anything.

How we built it

Five layers:

1. Sanitizer: removes hidden HTML and invisible text.
2. Detector: marks an email as safe, suspicious, or malicious (ML + rules).
3. Extractor: keeps trusted data (sender, subject) apart from untrusted email text, and marks attacker-controlled values as "tainted".
4. Policy engine: checks every tool call the model wants to make. It blocks calls that use tainted values or send to addresses that are not allowed.
5. Approval gate: asks the user before risky actions. The default answer is No.

The agent runs on NVIDIA Nemotron-3-Nano-30B-A3B through Nebius Token Factory. A web demo shows Baseline (no defense) and Protected side by side, with an audit log.

Results (real model)

We ran 90 real runs: 2 rounds, 5 attack types, 3 modes, 3 runs each.

• Round 1: the model ignored our weak attacks and only used safe tools, so it tested nothing.
• Round 2 (stronger attacks):
  • No protection: attacks worked 13 of 15 times.
  • Policy engine only: 0 of 15.
  • Full protection: 0 of 15.

Challenges

• Our first attacks were too weak, so we had to redesign them.
• The policy engine blocked 4 safe search calls (false alarms).
• The detector misses many attacks (about 55% caught), so the policy engine does most of the real blocking.
• The demo badge can show the wrong label when the model copies a trailing dot in the attacker address. The audit log is still correct.

What we learned

Do not trust the model to say no. Check every tool call outside the model. Checking where a value came from (the "tainted" idea) worked better than guessing if an email looks bad.

What's next

Improve the detector, add an allowlist check before any forward or reply, and run at least 10 runs per test.n

Built With

• fastapi
• github
• nebius-token-factory
• nvidia-nemotron
• python
• render
• scikit-learn
• uvicorn