Inspiration
Protocol SIFT showed that AI can assist forensic investigations - but it also revealed two critical flaws: hallucinated findings that analysts can't trust, and safety guardrails that exist only as prompt instructions the model can ignore. We asked: what if safety and accuracy weren't bolted on after the fact, but enforced by the architecture itself? What if a forensic AI physically couldn't fabricate evidence or modify data, not because we told it not to, but because the capability doesn't exist?
What it does
SIFT Sentinel autonomously investigates disk images using a dual-agent loop. An Analyst agent examines evidence through 15+ read-only forensic tools and produces findings with artifact-level citations. A Skeptic agent independently re-verifies every citation by re-executing the same tool calls. If a finding lacks evidence or contradicts the data, it's rejected and sent back for revision. The system produces a narrative report where every claim traces back to a specific byte offset, registry key, or MFT entry - and an execution log that proves it.
How we built it
Architecture-first. We designed the MCP server contract before writing any agent code - every tool is read-only by construction, returns structured results with citations, and enforces path validation and output bounds. Then we wrapped Sleuth Kit and regipy into typed Python functions, built the Coordinator (pure Python, no LLM) to orchestrate the loop, and used Pydantic schemas with min_length=1 on evidence lists so findings physically can't exist without citations. We validated against the M57-Jean forensic image from Digital Corpora and built a ground-truth benchmark harness to measure precision and recall.
Challenges we ran into
The M57-Jean image is Windows XP era (2008) - no AmCache, no EVTX, no modern Prefetch format. We had to build tools that fail gracefully on missing artifacts while still extracting value from what's available. Calibrating the Skeptic was hard: too strict and nothing passes, too lenient and hallucinations slip through. Forensic tool output is messier than expected - registry parsers and MFT tools produce subtly different formats across OS versions, requiring a normalization layer. And constraining subprocess calls to prevent shell injection while still wrapping 10+ command-line forensic tools took careful engineering (whitelist of binaries, programmatic argument arrays, no shell=True anywhere).
Accomplishments that we're proud of
Zero spoliation events across 5 adversarial attacks - shell injection in filenames, prompt injection in evidence records, direct deletion requests, malicious YARA rules, and fake admin overrides. All blocked because the capability doesn't exist, not because a prompt says "don't." The Skeptic caught real errors in production: timestamp mismatches, incorrect MITRE ATT&CK mappings, CONFIRMED/INFERRED conflation, and unsupported characterizations. Every single accepted finding traces back to a specific tool execution with a verifiable output hash. And the architecture is extensible - adding memory analysis requires only a .mem file and new tool implementations; the safety guarantees carry over automatically.
What we learned
Architectural constraints beat prompts every time. We tested prompt injection attacks that would bypass any instruction-based guardrail - they all fail silently because the write/delete/shell capability simply doesn't exist in the tool surface. Schema enforcement is underrated: making citations structurally required (not encouraged) eliminated all fully-fabricated findings. The Skeptic-Analyst loop provides genuine value - it caught errors that would pass any single-agent review. And honest reporting matters more than perfect numbers: our 14.3% hallucination rate reflects misinterpretation of real artifacts, not fabrication, and documenting that distinction is more credible than claiming perfection.
What's next for SIFT Sentinel
Memory analysis integration - adding a case with a paired memory dump to activate the volatility3 tools and disk↔memory correlation (flagging deleted droppers, LOLBin abuse). Support for modern Windows 10/11 images where AmCache, EVTX, and full Prefetch provide richer coverage. Cost optimization by experimenting with model allocation between Analyst and Skeptic. And publishing the benchmark harness as a standalone tool so the DFIR community can contribute ground-truth manifests for additional cases.
Log in or sign up for Devpost to join the conversation.