golem

You built them. Are you watching them?

The Problem

Last year, 71% of organizations got breached through identity. 41% of those breaches didn't start with a person. They started with a service account, an API token, or a CI/CD bot that nobody was watching.

The issue is that every behavioral analytics tool, including Splunk UEBA, was built around human patterns. Logins at weird hours, access from new countries, that sort of thing. But service accounts aren't humans. A deployment bot running at 3am is fine. A data pipeline making thousands of API calls per minute is doing its job. Run human anomaly logic on machine identities and you either drown in noise or miss the actual attack.

Splunk deprecated standalone UBA in December 2025. The replacement is ES Premier only. So most Splunk Enterprise and Cloud users currently have no native way to treat machine identities as their own category.

That's what Golem does.

---

How AI Is Used

Golem is a Python agent that talks to Splunk entirely through the Splunk MCP Server (Splunkbase ID 7931), over its Streamable HTTP transport. Every five minutes it:

1. Runs splunk_run_query to pull aggregated auth and cloud activity for every NHI it knows about.
2. Compares that against a 30-day per-hour behavioral baseline, stored in a KV Store collection.
3. Runs 26 detectors across that data, each targeting a specific behavioral signal for machine identities.
4. For every anomaly, calls saia_generate_spl to get a ready-to-run investigation query and saia_explain_spl for a plain-English summary, then asks the Foundation AI Security Model to classify severity (Low/Medium/High/Critical) and explain why this particular behavior is suspicious.
5. Writes the result to Splunk as a Notable Event, and powers a live "Ask Golem" panel via saia_ask_splunk_question, grounded in the selected identity.

For volume spikes specifically, Golem trains a real Splunk ML Toolkit DensityFunction model per identity:

| fit DensityFunction hourly_calls by Account_Name into golem_volume_density

over 30 days of hourly samples, then runs | apply golem_volume_density every cycle to get a genuine IsOutlier(hourly_calls) verdict. No fixed z-score thresholds, no GPU needed.

Golem adapts to whatever AI capabilities your Splunk deployment exposes. We tested the full pipeline against two real instances:

• A local Splunk Enterprise install with MCP Server, where Golem builds baselines, trains the MLTK model, runs all 26 detectors, and writes Notable Events via real MCP tool calls.
• A Splunk Cloud Platform trial, connected over the MCP Server's HTTPS endpoint, where splunk_run_query runs cloud-side and the full SAIA toolset (saia_generate_spl, saia_explain_spl, saia_ask_splunk_question, saia_optimize_spl) handles investigation queries and explanations.

Where a hosted AI capability isn't available on a given instance, Golem falls back to local equivalents: deterministic severity scoring and evidence-derived explanations. The dashboard, investigation workflow, and "Ask Golem" panel work the same either way.

---

What Golem Detects

26 detectors across five categories, all built on a per-NHI 30-day behavioral baseline:

Access and location New source host, new destination endpoint, lateral movement (3+ hosts in 15 min), cross-account access, new AWS region, new user agent, new access key used, impossible travel.

Timing and volume Unusual hour, volume spike (3σ + MLTK DensityFunction), API action diversity spike, data egress spike, bulk delete spike, off-hours sensitive action.

Privilege and persistence Privilege escalation, new role assumed, credential persistence (new access key created), MFA disabled, scheduled task creation, console login on a service account.

Attack signals Failed auth spike, access denied spike, secret access anomaly (Secrets Manager / SSM / KMS), security control disabled.

Identity hygiene Shadow NHI (active but never baselined), dormant reactivation (identity that went quiet, now suddenly active again).

---

How It Works

Every detection becomes a Notable Event with:

• NHI name, the specific account, token, or bot
• Anomaly type, which of the 26 patterns fired
• Risk score, Low / Medium / High / Critical
• AI explanation, from Foundation AI Security Model, in plain English
• Investigation SPL, generated by saia_generate_spl, ready to run

The point is to give analysts a short, prioritized list of machine identities doing something new, each one already explained and already investigable. Not a wall of alerts.

---

Deployment

Golem runs as a lightweight Python agent next to your Splunk instance, local Enterprise or Cloud, both verified. Install the MCP Server, optionally the AI Toolkit and the Python for Scientific Computing add-on for the MLTK model, drop your token in .env, run the agent. Done.

Built With

• foundation-ai-security-model
• python
• splunk
• splunk-ai-toolkit
• splunk-cloud-platform
• splunk-enterprise
• splunk-mcp-server
• splunk-ml-toolkit-(densityfunction)
• splunk-sdk-for-python