Inspiration
The thing that broke our brains was the GTG-1002 disclosure. An AI-orchestrated campaign running at 80-90% autonomy, full domain control in under 8 minutes, across dozens of networks simultaneously. Human analysts with manual CLI workflows cannot close that gap. That part we already knew.
What we hadn't fully sat with is what happens when you naively throw an LLM at the problem. An AI that hallucinates a malicious artifact into a forensic report doesn't just produce a false positive. It potentially tanks a court case. Daubert challenges are real, and opposing counsel will absolutely argue that an autonomous agent contaminated the evidence. You've traded one catastrophic problem for a different one.
The question that drove A.C.E. was: can you build something that operates at machine speed AND is rigorous enough to survive a federal evidentiary hearing? The answer isn't a smarter prompt. It's a different architecture entirely.
What it does
A.C.E. is an autonomous multi-agent DFIR framework built on Protocol SIFT and the Model Context Protocol. The core rule: no finding gets confirmed from a single source.
When the primary agent detects a suspicious process in memory, A.C.E. automatically dispatches secondary agents to cross-validate against independent disk and registry artifacts before anything gets marked confirmed. If the MFT says a file was created at 10:00 AM but the USN Journal shows activity at 09:55 AM, A.C.E. flags a timestomping anomaly without any human input.
Every output is explicitly labeled UNVERIFIED_INFERENCE or
CONSENSUS_CONFIRMED, and every label is traceable back to the exact tool
execution that produced it. The agent doesn't get to hide behind vague
conclusions.
It also defends itself against adversarial artifacts. Before any forensic output enters the LLM's context window, an Artifact Firewall scans for embedded prompt injection patterns and replaces suspicious strings with opaque UUIDs. The LLM reasons about behavioral patterns on sanitized data; the original strings are restored only in the final human-readable report.
How we built it
Architecture: Custom MCP Server (Pattern 2 from the hackathon framework)
The LLM gets zero shell access. Three typed MCP servers handle all tool execution:
volatility3MCP Server: exposesget_pslist(),get_netscan(),get_malfind(). Returns structured JSON, not raw terminal output.plasoMCP Server: exposesget_timeline(image_id, start_ts, end_ts)with hard-bounded output pagination to prevent context rot on large timeline dumps.tskMCP Server: exposesget_mft_entry(inode)andget_usn_journal(start_ts)for disk-level cross-validation.
The orchestrator runs a three-phase consensus cycle. Triage first: the primary
agent scans memory and disk, generates hypotheses with explicit reasoning fields
(hypothesis, selected tool, expected outcome). Cross-validation second: secondary
agents independently query corroborating artifacts with no access to the primary
agent's hypothesis until after they've run. Adjudication third: findings that
fail corroboration stay UNVERIFIED_INFERENCE and route to the human review
queue rather than getting silently promoted or dropped.
Forensic soundness is architectural, not instructional. Evidence volumes are mounted read-only at the OS kernel level. A T-0 SHA-256 hash of all evidence is generated before the agent is granted any access. An append-only JSONL audit log captures every tool call, exact parameters, output hash, token count, and timestamp. A hard iteration cap prevents infinite agent loops.
Challenges we ran into
Context rot was the first wall. Feeding a full Plaso timeline into an LLM
context window breaks coherence fast, and the agent starts producing confident
nonsense around the 40-minute mark of a large dataset. The fix was pagination at
the MCP server layer: each timeline call returns a bounded slice with a
next_page_token, so the agent processes chronologically rather than drowning
in one massive dump.
Confirmation bias in the primary agent was subtler. The first prototype let the triage agent run all its own corroboration checks. It consistently found supporting evidence for whatever it hypothesized first. The fix was architectural separation: secondary agents have no access to the primary's hypothesis until after they've independently queried their artifacts.
Prompt injection tuning took more iterations than expected. The Artifact
Firewall heuristics had to distinguish between a malicious username containing
injection patterns and a legitimate file path containing the word system.
Over-sanitizing legitimate forensic strings is its own failure mode.
Accomplishments that we're proud of
Evidence spoliation is architecturally impossible in A.C.E., not just discouraged by a system prompt. That distinction matters enormously in a legal context, and it's the difference between a demo tool and something a DOJ forensic lab would actually consider deploying.
The Artifact Firewall catching live prompt injection during testing against a
maliciously crafted username was a satisfying moment. The agent completed its
behavioral analysis on the sanitized <User_ID_042> object and correctly
identified the activity as anomalous, completely insulated from the adversarial
payload embedded in the artifact.
The audit log doubling as a training artifact was an emergent property we didn't fully anticipate. A junior analyst reading the adjudication log can trace exactly which corroborating evidence promoted or rejected each hypothesis, which is effectively a worked example from a senior analyst's investigative methodology.
What we learned
The difference between a demo and a production DFIR tool is almost entirely about failure mode documentation. A tool that hallucinates cleanly looks impressive in a five-minute video. A tool that flags its own uncertainty, labels findings explicitly, and degrades gracefully to human review is what actually gets deployed.
Building the consensus mechanism also forced a clear answer to a question we'd been hand-waving: what is a "finding" in AI-assisted forensics? It's not an LLM output. It's a claim backed by independently sourced, deterministically generated artifact evidence. That definition changed how we structured everything downstream.
What's next for Project A.C.E. (Artifact Consensus Engine)
- PCAP integration via Zeek to extend cross-artifact correlation to network traffic, enabling timeline correlation across memory, disk, registry, and network layers simultaneously.
- Neo4j temporal graph backend for multi-host cases: ingesting timelines from multiple endpoints and running cipher queries to detect temporal impossibilities (same credential authenticating in two geographic locations within a two-second window).
- Open-sourcing the Artifact Firewall as a standalone middleware library, decoupled from the rest of A.C.E., so any MCP-based DFIR tooling can drop it in as a sanitization layer.
- CI/CD benchmarking harness against the Digital Corpora public datasets (M57 patents scenario, Nitroba University case) to produce precision/recall metrics across LLM model versions and detect when a model update degrades forensic accuracy rather than improving it.
Log in or sign up for Devpost to join the conversation.