Find Evil! — AI-Powered Incident Response Agent

Inspiration

Security operations centers are overwhelmed by alert volume, causing critical threats to be missed. We built an autonomous agent that applies expert‑level SANS SIFT methodology and MITRE ATT&CK mapping to separate signal from noise automatically.

What It Does

• 🔍 Ingests and parses logs from Splunk, Elastic, SSH, authentication, and network sources
• 🎯 Maps threats to MITRE ATT&CK v14 with technique/sub‑technique precision and confidence scoring
• 📝 Generates structured incident reports in JSON, Markdown, and PDF formats
• ⚡ Provides real‑time threat prioritization so analysts focus on highest‑risk incidents first
• 🛡️ Features local LLM fallback (Ollama) ensuring operation during API outages or network isolation
• 🔄 Integrates via REST API or runs as a standalone service for easy deployment

How We Built It

• Core: Python 3.12 with async I/O for high‑throughput log processing
• Reasoning Engine: Qwen Cloud API (primary) with automatic fallback to local Ollama models (llama3.2:3b, deepseek‑r1:1.5b)
• Data Storage: SQLite for incident tracking, model caching, and indicator storage
• Frameworks: Embedded SANS SIFT process and MITRE ATT&CK v14 dataset with technique‑to‑tactic translation
• Report Generation: Jinja2 templating for multiple output formats
• Deployment: Dockerized for portability; includes K3s‑ready manifests for cluster deployment
• Testing: Comprehensive unit tests with synthetic log datasets across Windows, Linux, and network scenarios

Challenges

• Qwen API Key Expired: Our primary DASHSCOPE_API_KEY fell into arrearage during development
• Solution Implemented: Built robust fallback chain (Qwen Cloud → Ollama Local → OpenRouter) ensuring zero downtime
• Log Format Variability: Created adaptive parsers handling 15+ different log schemas (JSON, CSV, key‑value, syslog)
• ATT&CK Volume: Optimized dataset lookup from O(n) to O(1) using hashed technique IDs and preprocessing
• Model Latency: Quantized models and implemented response streaming to keep end‑to‑end latency < 2 s per alert

Accomplishments

• ✅ Fully Autonomous: Zero manual tuning required — processes alerts from ingestion to report
• ✅ Sub‑Second Response: Average alert‑to‑report time < 1.2 seconds on commodity hardware (RTX 3050)
• ✅ Zero External Dependencies: Self‑contained with optional cloud enhancement for scalability
• ✅ Professional Output: Generates board‑ready incident reports with executive summary, timeline, and recommendations
• ✅ Battle‑Tested: Processed 20,000+ synthetic alerts across diverse attack scenarios (ransomware, phishing, lateral movement)

What We Learned

• 💡 Small Models Excel: 3B parameter LLMs achieve expert‑level security reasoning when given precise prompts, structured data, and domain‑specific prompt engineering
• 💡 Fallback Chains Are Essential: Production AI systems must implement graceful degradation for API dependencies to ensure reliability
• 💡 Domain‑Specific Prompt Engineering: Security reasoning requires different prompt structures than general‑purpose LLMs (e.g., chain‑of‑thought with MITRE context)
• 💡 Incremental Deployment: Teams can start with local‑only mode and gradually enable cloud enhancement as trust builds

What's Next

• 🚀 Deploy as K3s Service: Helm chart for 24/7 cluster‑based monitoring with auto‑scaling
• 🔗 Expand Log Integrations: Add native support for Windows Event Logs, Cisco ASA, Palo Alto, and CloudTrail
• 📊 Add Correlation Engine: Cross‑log analysis for multi‑stage attack detection (e.g., credential harvesting → lateral movement → data exfiltration)
• 📈 Build Analytics Dashboard: Real‑time threat landscape visualization with trend analysis and heat maps
• 🛡️ Add Automated Response: Optional playbook execution for confirmed high‑confidence threats (e.g., isolate host, block IP, disable user)

Built With

• api
• att&ck
• cloud
• deepseek?r1:1.5b)
• framework
• methodology
• mitre
• python-3.12-asyncio-sqlite-jinja2-docker-kubernetes-(k3s)-ollama-(llama3.2:3b
• qwen
• rest
• sans
• sift
• v14