Inspiration
AI agents don't "do" things. They emit MCP tool calls, and a SIEM logs each call as its own event. The problem is that an attack on a hijacked agent isn't one event; it's a causal chain. One tool reads a credentials file, the next tool POSTs that exact content to an external host, a third beacons out. Splunk faithfully records those as three unrelated, low-severity lines. The thing that makes them an attack, the output of one call flowing into the arguments of the next, is never written down anywhere. A Tier-1 analyst reconstructs it by hand, about 30 minutes per incident, if they catch it at all.
I wanted to materialize that causal link as first-class data inside Splunk, and let a model, not a human, read the intent.
What it does
MCP Provenance Ledger reads the MCP tool-call stream and, within each agent session, draws a data-flow edge whenever one tool's result reappears in the next tool's arguments. Those edges form a provenance graph, which is scored, then judged and narrated by a local security LLM with MITRE ATT&CK technique IDs. Three unrelated alerts become one causal kill chain, surfaced in a native Splunk app with a custom provenance-graph visualization, a filterable case list, detection analytics, and a per-case detail page.
How I built it
It's a two-layer pipeline, reproducible end to end with no fixtures. python3 tests/eval.py runs it live and asserts every number.
Stage 1: structural recall, in pure SPL. The detector is a saved search. The subtle part: MCP results arrive as {"content":[{"type":"text","text":"…"}]}, so a naive substr of the raw field only ever matches the constant prefix {"content":[. I spath-extract the inner text first, then run a windowed value-flow test: does a 16-character window of one tool's extracted result reappear in the next tool's arguments? That's a genuine data dependency, not a "reached-external" heuristic. Each session is scored:
$$\text{killchain_score} = \text{edges} + 2 \cdot \text{external}$$
where edges counts real value-flow links and external marks a call that left the network.
Stage 1.5: MLTK corroboration. A OneClassSVM (Splunk Machine Learning Toolkit) flags structural outliers as an independent, model-driven signal.
Stage 2: precision, local LLM, $0. bin/confirm.py hands each Stage-1 candidate to Foundation-Sec-8B running locally via Ollama: judge it (kill chain vs. benign telemetry), narrate it, map it to ATT&CK, and write only confirmed chains, with verdict, confidence, and narration, to the KV store. The same logic can route through a Splunk-hosted model via the AI Toolkit | ai command instead, so the operator chooses sovereign on-prem or Splunk-managed.
The app renders all of it natively (Overview, Kill Chains, Detection Analytics, and a per-case Detail page with the provenance graph), plus a live attack replay (bin/agent_sim.py) that prompt-injects a simulated agent and catches it end to end in about 12 seconds.
The hard part: making the LLM load-bearing, not decoration
The honest failure mode of every "just blocklist the external hosts" approach: real attacks exfiltrate through the same reputable services as legitimate agents (S3, Slack webhooks, gist, pastebin). So a hostname allow/block-list catches only 16 of 60 attacks, and would also alarm on the benign sessions using those same hosts. Structurally, attacks and benign telemetry overlap completely at score 3. The only thing that separates them is the sensitivity of the data flowing through the chain, which a security-pretrained model reads and a regex cannot. Stage 1 buys recall by honestly over-flagging; Stage 2 buys precision by clearing 69 of 70 false positives.
Challenges I ran into
- The JSON envelope. Half a day gone before I realized the raw-field substring was matching a constant prefix, not the data.
spath-extracting the inner text first was the unlock for the whole detector. - Proving the LLM earns its place. I built a hard-benign telemetry set (legit agents that read-then-post-externally, the exact case that breaks naive detectors) specifically to try to break my own system, then measured the blocklist baseline (16/60) to show the LLM isn't cosmetic.
- Honesty over a clean number. The one residual false positive (internal metrics posted to a public pastebin) is genuinely ambiguous, the kind of thing a DLP-conscious SOC would want flagged. I reported it instead of tuning it away.
- Sovereign and $0. The model runs on-device via Ollama; no telemetry leaves the host.
Accomplishments that I'm proud of
- 100% recall, 98% precision, 1 false positive in 470 benign sessions, a 99% cut from the structural baseline, measured live with no fixtures.
- A native, in-Splunk custom visualization that draws each kill chain as a data-flow node graph with ATT&CK chips and a verdict.
- The whole thing is reproducible:
tests/eval.pymaterializes Stage 1, runs Stage 2 for real, and asserts the metrics. Nothing is hardcoded.
What I learned
- MCP tool-call telemetry is a genuinely rich provenance source for the agentic SOC. The value-flow between calls is the signal everyone else is throwing away.
- Splitting recall and precision across a structural layer and an LLM layer is cleaner and more defensible than forcing one detector to do both.
- A small, security-pretrained local model can do real SOC triage at zero cost and zero data egress.
What's next for MCP Provenance Ledger
- Real-time streaming detection wired to Notables and SOAR playbooks.
- Broader ATT&CK coverage and more agent frameworks.
- Resolving the residual false positive with DLP content signals.
- Production hardening: bounded LLM concurrency for high-volume MCP streams.
Built With
- fastapi
- foundation-sec-8b
- javascript
- mitre-att&ck
- model-context-protocol-(mcp)
- ollama
- python
- react
- react-flow
- spl
- splunk-ai-toolkit
- splunk-custom-visualizations
- splunk-enterprise
- splunk-kv-store
- splunk-machine-learning-toolkit-(oneclasssvm)
- vite