Phishnet AI

A Splunk app that runs real phishing investigations on every alert, shows its work, and tells you which ones actually executed on an endpoint.

Comment

Phishnet AI is a Splunk Enterprise app that does the work a Tier-1 analyst would do on a phishing ticket: check sender and URL context, figure out who got the mail, look for clicks and credential submissions, and correlate endpoint telemetry to see if anything actually ran.

It runs as a modular input inside Splunk. Alerts land in index=phishing. The agent walks each one through a fixed playbook, calls Foundation-Sec-8B for a verdict and short reasoning, and writes the result to phishnet_decisions in KV Store. Dashboards read from that collection, not from ad hoc index searches, which turned out to be important once I had 300 alerts in the demo queue.

Analysts get a Command Center with the full queue, a reasoning drilldown per alert, and an end-of-shift handoff they can export to CSV. Blast Radius ties mail events to host metrics so you can answer a concrete question: did the payload execute, and on which machine? Managers get a separate ROI view with throughput and hours saved, gated behind a phishnet_manager role so analysts are not staring at leadership metrics during triage.

The same pipeline is exposed over MCP. External clients can call investigate_alert or triage_queue the same way the modular input does. When the official Splunk MCP Server is installed, five independent searches per investigation run in parallel through splunk_run_query, with the Python SDK as fallback if a call fails.

My demo data is a 300-alert overnight shift. The agent auto-closes most false positives, surfaces six targeted attacks, and PH-0286 is the case I walk through on video: credential theft and confirmed payload execution on WKSTN-904.

Repo: https://github.com/VineetLoyer/phishnet Video: https://youtu.be/PBy3xOAFcZo

Inspiration

The starting point was boring and familiar: a SOC queue with hundreds of phishing alerts and not enough time to open all of them properly. A real investigation is twenty to thirty minutes if you do it right. In practice people batch-close noise, skim subjects, and hope the dangerous one is not buried at row 187.

I wanted automation that actually investigates, not automation that hides the queue. That meant two things from day one. First, every alert gets the same playbook, not a shortcut for "probably fine" mail. Second, the agent has to show its steps in Splunk so an analyst can disagree with it. Black-box auto-close is how you lose trust in week one.

There is a second audience I kept in mind: whoever owns the SOC budget. They hear "AI" and want proof. Phishnet writes structured decisions back into Splunk, so throughput, false-positive rate, and time saved are queryable from the same place the analysts already work. I did not want a separate reporting product glued on the side.

How I built it

Everything ships as the phishnet_ai Splunk app. phishnet_agent.py is the modular input: poll alerts, run the pipeline, write KV and index events. phishnet_remediate.py is the alert action when an analyst approves remediation. Shared logic lives in phishnet_lib: investigation steps, classifier adapters, threat intel, orchestrator, and shift handoff.

The investigation playbook is deterministic Splunk work first, model second. I gather signals (reputation, exposure, user interaction, endpoint context), pass them to the classifier as structured context, and persist both the verdict and a step-by-step trace in steps_text for the UI. That ordering matters. The model does not replace the playbook; it interprets what the playbook found.

Threat intel is computed from data already in Splunk: alert volume and recipient spread in index=phishing, prior verdicts in phishnet_decisions, cached in phishnet_threat_intel. I started with external VT/urlscan stubs and dropped them. For a Splunk hackathon and for most enterprise installs, "your own history plus your own telemetry" is a more honest story than API keys you do not have in the demo environment.

Dashboards were a lesson in Simple XML pragmatism. Panels that searched indexes directly looked fine with ten events and empty with three hundred. I standardized on | inputlookup phishnet_decisions and mirrored audit events into phishnet_audit_log KV for the activity feed.

MCP shows up twice. I run phishnet_mcp_server.py (FastMCP, stdio) so IDE agents can drive the pipeline. Separately, splunk_mcp_client.py talks to Splunkbase app 7931: mint a token, call splunk_run_query, fall back to splunklib on failure. The orchestrator fans out five searches with a thread pool and records wall time versus sequential SDK calls. On the hero alert that was roughly 5x faster, which is nice for a live demo but also how I would structure production fan-out.

RBAC is small but intentional. phishnet_analyst sees Command Center and Blast Radius. phishnet_manager adds Manager ROI via metadata/default.meta. Analysts should not see manager navigation entries they cannot use.

The end-of-shift handoff went through three iterations: mailto link with a dashboard URL, a REST endpoint with inline JavaScript, and finally a KV-backed table with CSV download. The last one survived contact with real Splunk UI behavior.

Packaging for Splunkbase is app.manifest plus scripts/package_app.ps1, which emits dist/phishnet_ai-1.0.0.tar.gz.

Challenges I ran into

Dashboards that lie. Simple XML panels bound to indexes returned empty rows intermittently at demo scale. The fix was not more SPL in the panel; it was making KV the source of truth after the agent writes once. That added schema work up front but removed an entire class of "works on my laptop" failures.

300 alerts exposed naive I/O. Early enrichment hit KV once per indicator. Repopulating the demo took forever. I batch-seed reputation, preload the intel cache in process memory, and treat KV as write-through storage, not a per-step lookup service.

Splunk MCP auth is not a bearer header you paste from a doc. The official server expects RSA-encrypted tokens minted through Splunk's REST API. I wrote a thin client around that and kept SDK fallback so a failed MCP call does not abort an investigation mid-playbook.

Language drift between model output and security reality. Foundation-Sec flagged six targeted_attack verdicts in the demo set, but only PH-0286 had endpoint execution evidence. The UI had to say "targeted attack" where the model did and reserve "payload executed" for telemetry-backed cases. Conflating those makes analysts ignore the dashboard.

Handoff in the browser is harder than it looks. Splunk Simple XML and ad hoc JavaScript do not mix cleanly. Entity escaping broke my first panel; a REST handler broke the second. The version that worked is boring: a table generated from SPL, CSV export from client-side JavaScript reading that table. Boring was the point.

Accomplishments I'm proud of

I shipped a complete Splunk app, not a script and a slide deck: modular input, alert action, KV schemas, three dashboards, RBAC, audit trail, pytest, and smoke scripts that run against live Splunk.

The dual MCP setup is real on both sides. I expose tools and consume Splunk's server, with measured parallel speedup on investigation searches.

PH-0286 is the narrative I would actually tell in a SOC handoff: reasoning you can read, blast radius you can verify, payload execution on a named host. The aggregate stats (300 processed, 240 closed) matter, but the hero case is what makes judges and analysts believe the rest.

What I learned

Splunk's MCP server is a good fit for agent fan-out. SPL stays SPL; you do not invent a parallel query API. You still need fallback paths because network calls fail.

Reputation from your own indexes is easier to demo and defend than pretending everyone has VirusTotal wired on day one.

KV Store is underrated for Simple XML dashboards. If the UI reads what the agent wrote, panels stop fighting the search scheduler.

recommend mode was the right default. Auto-close everything and analysts will turn the agent off. Show the reasoning, let them confirm the easy ones, and automation earns room to run.

One well-investigated alert on video beats ten bullet points about "AI-powered synergy." I structured the demo around PH-0286 on purpose.

What's next for Phishnet AI

CIM and ES email field alignment so installs with Enterprise Security get native correlation. Predictive blast radius using Splunk's time-series tooling instead of static thresholds. An educational mode that walks junior analysts through why a step was fired, not just what the verdict was.

Built With

  • foundation-sec-8b-via-ollama
  • mcp
  • model-context-protocol-/-fastmcp
  • python-3.11
  • shell
  • splunk
  • splunk-enterprise-10.x
  • splunk-kv-store
  • splunk-mcp-server-(splunkbase-7931)
  • splunk-python-sdk
  • xml
Share this project:

Updates