EvidenceOps Agent

Inspiration

AI-speed attacks make manual incident response too slow. EvidenceOps Agent was built to show how a defender can move from scattered evidence to a traceable response brief without accepting black-box AI claims.

What it does

The FIND EVIL edition adds a terminal-running incident response agent in findevil/. It reads a local OAuth token theft evidence case, runs tool-style analysis steps, maps every finding to evidence IDs, records a structured execution log, and places disruptive response actions behind human approval gates.

The key judging behavior is self-correction. The agent initially considers a confirmed exfiltration claim, validates the storage artifact, sees that external destination proof is missing, and downgrades the finding to likely data exfiltration. That correction is written into findevil/execution_log_sample.json.

How we built it

The project has two judge-facing paths:

1. A web analyst workspace on GitHub Pages with incident scenarios, evidence timelines, reasoning cards, approval gates, and downloadable briefs.
2. A Python terminal agent that runs with only the standard library:

cd findevil
python3 agent.py evidence/oauth_token_theft_case.json --log execution_log_sample.json

Supporting materials include FIND EVIL-specific dataset documentation, an accuracy report, terminal demo script, architecture notes, and execution logs.

Challenges

The hardest part was avoiding overclaiming. The sample evidence proves suspicious OAuth activity and a large archive download, but it does not prove an external destination. The agent therefore has to recognize the gap and correct its own language.

What we learned

A useful IR agent is not just a summarizer. It needs evidence IDs, audit logs, uncertainty handling, and approval boundaries. Judges should be able to trace every major claim back to the artifact that produced it.

What's next

The next version should run directly inside SIFT/Protocol SIFT against richer disk, memory, log, and network artifacts. It should also add more self-correction cases and a longer benchmark of hallucinated claims caught before reporting.

Built With

• ai-agents
• css
• cybersecurity
• dfir
• forensics
• html
• incident-response
• javascript
• mcp
• python
• sift