Inspiration

As a SOC analyst I am tired of going through all platform and write SPL across multiple indexes to get the infos I need, I believe this should be AI's job. So I am designing a native splunk app aim for this pain-point.

What it does

Automatically collects relevant data from Splunk MCP and third party platform, instruct possible next action and retrieve new data to help with investigation. In the future I can even adding splunk SOAR to automate some actions.

How we built it

Built with native splunk app using agentic system to call MCP tools and API. User will use a chatbot like UI to communicate with AI.

Challenges we ran into

I have pivoted at least 5 times. Claude Code/Codex are great agentic system since they have premium model and top-tier context management, but they are not native splunk app and won't be scaled to other team easily. In certain country/region where data sovereignty is also a reality issue when using agentic system. So my approach is a native splunk app and custom LLM provider, this can even be run locally with local LLM if needed.

Accomplishments that we're proud of

Shorten a ton of triage time and enhance investigation coverage with better quality, reduce MTTR drastically.

What we learned

Designing agentic system should consider scalability, deployment effort and user experience so this won't be just a demo but can be put into production to other teams.

What's next for Agentic Incident Commander

Generate reports and dashboard locally on splunk for better visibility.

Share this project:

Updates