How GHOST evolved
GHOST started as a question: can an AI agent do the first investigation of a security alert the way a skilled analyst would? Here is how it got from idea to what you can try today.
1. A working pipeline on a practice attack. I built the two-layer design first: fast detection rules scan every log event and raise a few alerts, then an NVIDIA Nemotron agent investigates each alert with tools (log search, IP checks, user context, MITRE ATT&CK mapping). On a synthetic attack, the first version found only 2 of 4 attack techniques. Rewriting the instructions and feeding the detection rules in as hints took it to 4 of 4.
2. Real attacks, and the first real failures (October 3). I moved to real recordings from the OTRF Security Datasets project, scored against their official labels. The first real attack scored 0/1: the agent reached the right verdict but wrote an unreadable report, so I added self-correction. My detectors also missed Mimikatz entirely, so I wrote LSASS and WMI detectors. The agent sometimes claimed techniques with no evidence, so I added a grounding check that rejects them. Result: 8/8 across 4 real attacks and a practice attack, but 1,312 seconds of agent time on NVIDIA's free endpoint.
3. Nebius, and checking the checker (October 5). Switching to Nebius Token Factory was a settings change and cut agent time to 147 seconds. Then I found a loophole in my own grounding check that let an unsupported technique through. Closing it overcorrected and rejected true claims, so I made the tools state exactly which techniques their evidence proves. Final: 8/8, zero unsupported claims, 178 seconds. Around the same time the dashboard went live on Streamlit, and I recorded the demo video.
4. Proving the GPU claim (October 11). To test NVIDIA RAPIDS, I scaled the logs to 5 million events. The first run never finished: two detectors re-scanned every log for each suspicious event. After rewriting them as table-wide joins, the GPU was only 1.5x faster, because some operations silently fell back to the CPU. After fixing those, an NVIDIA T4 ran all 8 detectors 8.9x faster than the CPU, with identical alerts.
5. Submitted, then made testable by anyone (October 12-13). After submitting, I focused on letting judges see GHOST work, not just read about it:
- Live Nemotron runs on the public site, with per-visitor and daily limits to protect the API budget.
- Upload your own logs: GHOST-format or raw Windows event logs, run through the same pipeline.
- Full MITRE ATT&CK knowledge base: all 706 techniques are searchable, but search results stay suggestions, never evidence.
- A redesigned console with a live attack map that replays the machines and connections in the logs.
The latest benchmark with all of this enabled: still 8/8, zero unsupported claims, 160 seconds, about 11 seconds per alert.
The biggest lesson: guardrails need testing as much as the AI does. A "0 rejected" score once hid a loophole, and finding it made the final results trustworthy.
Try it live: ghost-soc.streamlit.app
Log in or sign up for Devpost to join the conversation.