Praxis

The AI SOC analyst that connects scattered Splunk alerts into one verdict — and hunts the same attacker across every account.

Comment

Praxis — Hackathon Submission

Elevator Pitch

Five whisper-quiet Splunk alerts. One AI-correlated verdict. Praxis hunts the same threat across accounts and closes the loop back into Splunk.

About the project

Inspiration

SOC analysts drown in noise: roughly 95% of alerts are individually low-severity, so they get triaged in isolation and closed. But the real breach signal is often several quiet alerts across different disciplines — one login, one Wi-Fi association, one scheduled task, one file-server hop, one egress spike — each of which looks benign on its own and stays below any single alert's threshold.

We were also struck by Splunk's own Dubai Airports case study: 200+ access points, up to 20,000 simultaneous connections, and rogue/evil-twin Wi-Fi APs as a real, large-scale attack surface. That became the seed of our planted attack scenario and our cross-account correlation feature.

The "Agentic Ops" theme of this hackathon was the perfect excuse to ask: what if a team of specialist agents did the correlation work a human analyst never has time for — and did it deterministically, so the SOC can trust and explain every verdict?

What it does

Praxis takes Splunk from "5 separate low-severity alerts" to "1 high-confidence verdict with a reconstructed kill chain" — without a human opening 5 separate searches.

  • 5 specialist agents investigate a flagged user in parallel via LangGraph — Identity Analyst (impossible travel, MFA push-bombing), Lateral Movement (cross-protocol file-server access, rogue Wi-Fi APs), Exfiltration (DNS tunneling, high-volume egress to low-reputation destinations), Persistence (unsigned scheduled tasks), and a Devil's Advocate that actively hunts for exculpatory evidence (travel records, change tickets).
  • Rule-based ScoringClient maps concrete event fields to a severity and confidence — deterministic, zero LLM calls, zero hallucination risk.
  • Correlation Lead fans everything back in: it escalates to ACTIVE_INTRUSION only when $\geq 3$ independent agents flag HIGH/CRITICAL severity, builds a time-ordered kill chain across all findings, and folds in any Devil's Advocate dissent.
  • Campaign Hunter — our cross-user correlation layer. It runs cross-user SPL (stats / dc(user), deliberately without a per-user filter) to find one indicator of compromise — a rogue access point, a shared exfil destination, a shared persistence artifact — touching $\geq 2$ accounts, then re-runs the full 5-agent investigation for every affected user and merges the results into one CampaignVerdict.
  • Real-time React console streams every agent's findings live over Server-Sent Events, then the final verdict and kill-chain timeline.
  • Closed-loop alerting — a native Splunk custom alert action re-runs the whole pipeline when a Praxis - * saved search fires and writes the verdict back into Splunk as sourcetype=praxis:verdict, fully queryable alongside the original raw events.

Defining demo moment: 5 individually-low-severity alerts for one user → ONE active_intrusion verdict with a reconstructed kill chain.

Campaign Hunter demo moment: one rogue access point → two compromised accounts (j.okonkwo → active_intrusion, e.osei → suspicious) → ONE campaign verdict tying them together with a combined, per-user-tagged kill chain.

How we built it

  • Orchestration: LangGraph StateGraph fans out to all 5 agents in parallel (shared findings: Annotated[list[Finding], operator.add] reducer), then fans in to a single correlation_lead node. stream_case() wraps astream(..., stream_mode="updates") so the API can emit one SSE event per agent as it completes.
  • Splunk access: McpSplunkClient is the only Splunk interface used by any agent — a thin async wrapper over the Splunk MCP Server's (app 7931) JSON-RPC 2.0 / Streamable HTTP API. Every agent issues its own hand-written SPL scoped to index=main, the target user, and an earliest_time window.
  • Data: data/gen_scenario.py is a deterministic generator (random.seed(1337)) producing 252 events across 5 sourcetypes (praxis:auth, praxis:network, praxis:endpoint, praxis:egress, praxis:wifi), including a planted multi-stage attack for j.okonkwo (rogue Wi-Fi association → impossible-travel login → MFA fatigue → lateral movement → persistence → exfiltration) and a second account, e.osei, associating to that same rogue access point — the shared indicator Campaign Hunter correlates on.
  • Scoring: ScoringClient is intentionally rule-based — field-threshold checks (e.g. geo_velocity_kmh > 1000 with no travel_record on file) mapped directly to severity/confidence/rationale. We evaluated Splunk AI Toolkit hosted models and Anthropic Claude for this step and deliberately did not use them here, for speed, reproducibility, and zero hallucination risk during a live demo.
  • API + UI: FastAPI backend (GET /investigate/{user} SSE, GET /campaigns JSON) feeding a React 19 + Vite + TypeScript + Tailwind v4 console with live agent panels, a verdict/kill-chain view, and a Campaign Hunter tab.
  • Closed loop: splunk_app/praxis_alert_action/ is a custom Splunk alert action — when a Praxis - * saved search fires, praxis_investigate.py launches run_alert_investigation.py, which re-runs the full pipeline and POSTs the verdict to the HTTP Event Collector as sourcetype=praxis:verdict.

Challenges we ran into

  • MCP tool limitations: the saia_* tools (AI Assistant for SPL) and MLTK's | ai command were broken in our environment. We pivoted entirely to hand-written SPL per agent — which turned out to be a feature, not a workaround: it makes the whole pipeline deterministic and demo-safe.
  • Index hygiene: index=main accumulated duplicate test-ingestion batches during development, and | delete didn't behave as expected on our setup. We worked around this with tight earliest_time windows scoped to the planted scenario's timestamps rather than relying on index cleanup.
  • Cross-user SPL design: Campaign Hunter's stats/dc(user) queries had to find shared indicators without any per-user filter, then hand off exactly the right indicator value (BSSID, domain, task name) to re-trigger five full per-user investigations — getting that contract right between the cross-user agent and the per-user orchestrator took a few iterations.
  • Time pressure on the home stretch: with the deadline closing in, we made a deliberate call to keep every remaining feature rule-based rather than reach for new LLM integrations or app installs late — protecting determinism and demo reliability over last-minute scope.
  • Windows dev quirks: Splunk's own web port (:8000) collided with our documented API port, so the backend runs on :8800 locally with the UI pointed at it via ui/.env.local.

Accomplishments that we're proud of

  • Live, not mocked: every agent, the Correlation Lead, and Campaign Hunter are verified end-to-end against a real local Splunk instance through the MCP Server — not stubbed responses.
  • Fully explainable AI: every severity, confidence score, and kill-chain step traces back to a concrete Splunk event field and a documented rule — no black box, no hallucination risk, fully reproducible.
  • Campaign Hunter: going beyond "investigate one user" to "find the campaign hiding across accounts" — one rogue AP becomes a two-account active_intrusion campaign verdict with a single merged kill chain.
  • Genuinely closed loop: a Splunk alert can trigger Praxis, and Praxis writes its answer back into Splunk — index=main sourcetype=praxis:verdict is queryable like any other event.
  • Real-time UX: the SSE-streamed console makes the multi-agent investigation feel alive — analysts watch each discipline report in as it finishes, not after a long blocking wait.

What we learned

  • A multi-agent, fan-out/fan-in architecture (LangGraph) can deliver "agentic" value — parallel specialist reasoning, synthesis, dissenting views — without requiring an LLM in the critical path. Determinism and explainability are themselves a feature for security tooling.
  • Designing SPL that works at two scopes — per-user investigation and cross-user correlation — pays off: the same 5-agent pipeline becomes the unit of work for both a single alert and a campaign.
  • Real MCP integrations have real rough edges (broken saia_*/MLTK paths in our environment); building a thin, swappable client (McpSplunkClient) early made it easy to route around those without touching agent logic.
  • Tight deadlines reward protecting what already works — our last design decisions were about preserving the deterministic, explainable core rather than chasing new integrations.

What's next for Praxis

  • Response Agent — a 6th agent that turns a verdict (or campaign verdict) into a prioritized, deterministic remediation playbook (block a BSSID, block an egress domain, remove a scheduled task, force a password reset/MFA re-enrollment), written back to Splunk as sourcetype=praxis:remediation — closing the detect → investigate → decide → respond loop. (Models and rule engine are already drafted.)
  • SOC Wallboard — an aggregate, always-on view across all users and active campaigns, surfacing the highest-priority verdicts at a glance.
  • More campaign indicators — extend Campaign Hunter beyond rogue APs to shared exfiltration destinations and shared persistence artifacts already modeled in the dataset.
  • Optional LLM narrative layer — once AI Toolkit / saia_* integrations stabilize, add an opt-in natural-language summary on top of the deterministic verdict — narrative for humans, rules for trust.
  • Production hardening — authentication, multi-tenant index scoping, and outbound integrations (Slack/email/ticketing) for the remediation playbooks.

Built With

Share this project:

Updates