gemini-splunk-security-agent

A Gemini agent on Splunk MCP that triages a Splunk ES notable event end-to-end and visibly self-corrects when surface SIEM evidence gets overturned by deeper context.

Comment

Why this exists

Most high-urgency Splunk Enterprise Security notable events fired by heuristic correlation searches — binary executed from %TEMP%, base64 -EncodedCommand PowerShell, sysmon EventCode=1 on suspicious paths — are false positives caused by sanctioned IT maintenance running under a service account inside an approved change window.

A Tier-1 analyst who pages the on-call on every notable burns the team out by Wednesday. A Tier-2 analyst forms a working hypothesis fast, then revises it loudly when the threat-intel lookup, asset owner, change-management table, and SOAR playbook state all disagree.

gemini-splunk-security-agent does that walk for you. Hand it an incident ID and it works the case through the official Splunk MCP server tools (Splunk ES + Splunk SOAR + Splunk TI framework), end-to-end, citing notable IDs, correlation-search SPL, change IDs, asset owners, and SOAR rationale verbatim from the tool output.

What it does

The agent's system prompt forces a five-step workflow over the Splunk Security MCP tool surface:

  1. list_notable_events(status="new") — surface what's currently in the SIEM queue.
  2. get_notable_event(event_id) — pull the correlation search SPL, the surface indicators (process_path, parent_process), and the AD user.
  3. threat_intel_lookup(indicator) — look up the SHA256 / domain / IP across the Splunk TI framework feeds (mandiant, crowdstrike, abuseipdb, internal_ti). Return the binary signature record (signed_by, signature_valid, matches_known_chain).
  4. asset_lookup(host) — resolve the host to its Splunk ES asset record (owner team, criticality, AD user classification) plus the change-management lookup covering that host.
  5. get_soar_playbook_state(incident_id) — read the running Splunk SOAR playbook state and rationale.

After step 2 the agent MUST state an INITIAL HYPOTHESIS based ONLY on surface indicators. After step 5 it MUST state a FINAL VERDICT. If those disagree, it MUST emit a labeled SELF-CORRECTION paragraph naming the exact pieces of evidence that flipped the verdict.

The canned incident

INC-2026-0518-A-001. Two correlated high-urgency notable events fire on the same endpoint (WS-DEV-7184.corp.local):

  • NTB-2026-0518-1432-A — Binary executed from %TEMP% (update_helper.exe).
  • NTB-2026-0518-1430-B — Encoded PowerShell observed on the same host.

Surface evidence says malware. Walking all five tools reveals:

  • The binary's SHA256 is IOC-clean across mandiant/crowdstrike/abuseipdb/internal_ti.
  • The binary is signed by Microsoft Corporation with signature_valid=true.
  • The src_host is owned by IT Endpoint Management.
  • The AD user is DOMAIN\svc_endpoint_admin, a sanctioned service account.
  • An approved change window CR-2026-MAY-0517 covers this run.
  • A Splunk SOAR playbook (endpoint_maintenance_validate) is mid-run with rationale "no escalation needed."

The agent flips from TRUE POSITIVE to FALSE POSITIVE — sanctioned admin activity and emits the SELF-CORRECTION paragraph explaining the flip.

How we built it

  • Model: Gemini 2.5 Flash on Vertex AI
  • Agent runtime: google.adk.agents.LlmAgent + McpToolset (Google Cloud Agent Builder / ADK)
  • MCP: Splunk Security tool surface — bundled stub for demos, real-tenant-ready via env vars (SPLUNK_HOST, SPLUNK_TOKEN, SPLUNK_SOAR_TOKEN)
  • Surface: Streamlit dashboard on Cloud Run + a runner.ask() Python entrypoint
  • License: Apache 2.0, standalone repo created during the contest period

Live Vertex AI smoke test

Reproducible via scripts/smoke.py against the stub MCP. The latest live run passed all six verbatim-output checks:

[PASS] has INITIAL HYPOTHESIS section
[PASS] has SELF-CORRECTION section
[PASS] has FINAL VERDICT section
[PASS] lands on FALSE POSITIVE
[PASS] names svc_endpoint_admin verbatim
[PASS] names CR-2026-MAY-0517 verbatim

The agent quoted both correlation searches' SPL character-for-character, the change_id, the AD user, the asset owner team, and the SOAR rationale.

Bonus prizes targeted

  • Best Use of Splunk MCP Server ($1K) — the agent's entire tool surface IS the Splunk MCP server. Five tools (list_notable_events, get_notable_event, threat_intel_lookup, asset_lookup, get_soar_playbook_state) match the official server's shape, stubbed for demos and real-tenant-ready via one env-var swap.

Why this is substantially different from gemini-splunk-agent

This is my second submission to the Splunk Agentic Ops Hackathon. The first (gemini-splunk-agent) is in the Observability track and uses five different Splunk MCP tools (list_alerts, get_detector, list_indexes, run_search, run_observability_query) for production incident investigation. Different track, different tool surface, different verdict shape. The Security entry's killer feature is the labeled SELF-CORRECTION section that visibly flips TRUE POSITIVE -> FALSE POSITIVE — uniquely valuable in the SOC where false-positive fatigue is the day-one problem.

License

Apache 2.0. Standalone repo created during the Splunk Agentic Ops contest period.

Built With

  • agent-development-kit
  • apache-2
  • gemini
  • gemini-2-5
  • google-cloud-agent-builder
  • google-cloud-run
  • mcp
  • model-context-protocol
  • python
  • splunk
  • splunk-enterprise-security
  • splunk-mcp
  • splunk-soar
  • streamlit
  • vertex-ai
Share this project:

Updates