AgentGuard for Splunk SOC

Inspiration

Security teams are starting to use AI agents for alert triage, incident summaries, log correlation, and remediation suggestions. The dangerous part is that an agent can look confident while still taking unsafe action: closing the wrong alert, skipping evidence, overusing tool permissions, or mutating production systems without approval.

AgentGuard for Splunk SOC was built around one question: before a SOC agent is allowed to act, can we prove that it is safe enough?

What it does

AgentGuard is a reliability gate for agentic SOC workflows. It runs repeatable scenarios, scores an AI agent across five reliability gates, and produces evidence that says whether the agent should be promoted, routed to a human reviewer, or blocked.

For this Splunk version, the product focuses on Security operations:

• AI alert triage and investigation workflows.
• Evidence preservation before case closure.
• Review gates for risky containment or suppression actions.
• Splunk-ready evidence packets for analysts and judges.
• A companion Splunk app with saved searches, dashboards, and an alert-action handoff.

How Splunk is used

Splunk is treated as the operational source of truth. The architecture models alerts, logs, and case context flowing from Splunk and Splunk MCP Server into AgentGuard. AgentGuard then evaluates the proposed AI-agent action and returns a decision envelope that can be reviewed, stored, or blocked before execution.

The repository includes a Splunk companion app under splunk-apps/agentguard_ci_for_splunk, with saved searches, a demo dashboard, lookup data, and an alert action that turns Splunk events into AgentGuard review envelopes. CI validates the app with the Splunk Packaging Toolkit and Splunk AppInspect path.

How I built it

The system is built as a TypeScript reliability platform with a React/Vite dashboard, Node.js scenario runners, GitHub Actions evidence generation, and a packaged Splunk companion app. The live-local test suite covers enterprise agent categories including security, browser/RPA, data analysis, customer support, workflow, document compliance, and code repair.

The Splunk submission adds SOC-specific scenarios, a dedicated contest presentation mode, root architecture documentation, machine-readable evidence, and a public demo video.

Challenges I ran into

The hardest challenge was making the project more than a generic AI demo. A SOC tool has to be auditable. That meant every claim needed evidence: repeatable tests, explicit risk reasons, generated artifacts, and CI checks that a reviewer can inspect without trusting a slide deck.

Another challenge was packaging the companion app in a way that feels real to Splunk users. The submission includes installable app structure, saved searches, dashboard assets, and validation checks instead of only screenshots.

Accomplishments that I'm proud of

• 24 command-backed reliability scenarios.
• 14 live-local enterprise scenarios across 12 agent categories.
• 3 dedicated security/SOC routes.
• A review/block queue that turns agent risk into an operational decision.
• A Splunk companion app and evidence packet that make the integration concrete.
• Passing CI gates for the public repository.

What I learned

The biggest lesson is that AI-agent safety should be evaluated like release readiness. Capability is not enough. A production agent needs evidence, boundaries, traceability, and a clear escalation path when confidence is not sufficient.

What's next

Next, I would connect the review envelope to a live Splunk Cloud or Splunk Enterprise workflow, add richer analyst feedback loops, expand detection engineering scenarios, and benchmark multiple agent frameworks under the same reliability gates.

Built With

• github-actions
• node.js
• react
• splunk
• splunk-appinspect
• splunk-mcp-server
• typescript
• vite