logflip-sift-agent

Inspiration

State-sponsored and AI-driven attacks now execute at machine speed: reconnaissance and exploitation in minutes. Incident response has to answer in seconds, and that pressure makes autonomous IR agents tempting. But in forensics, an agent that hallucinates a finding is worse than no agent at all: a fabricated verdict can misdirect an investigation, burn an analyst's time, or taint evidence.

SANS Protocol SIFT draws the line clearly: the AI should direct verified tools and self-correct, not interpret raw bytes or decide verdicts. I wanted to prove that autonomy and forensic integrity are not in tension, as long as the guarantees that protect the evidence are architectural and not just a prompt the model could ignore. NTFS timestomping was the perfect target: subtle, high-stakes, and detectable across the filesystem's own journals.

What it does

logflip-sift-agent is an autonomous agent that finds NTFS timestamp tampering on a disk image and reasons about each candidate the way a senior analyst would.

It scans the $LogFile for records whose in-image timestamps disagree with the journal, investigates each one by reverse-replaying the journal to recover a signed verdict, and, when a record looks tampered but has no journal coverage, self-corrects by corroborating across an independent channel before drawing a conclusion. Every finding is bound to an HMAC-signed evidence leaf and a timestamped execution log, so a reviewer can trace any claim back to the exact tool call and re-verify it offline.

On the reproducible demo case the run is unambiguous: records 5 and 7 are journaled stomps reported as provisional (each backed by a signed leaf), and record 12 is a single-source anomaly that is corroborated and then honestly not escalated. Exit code 2.

The key property: the agent cannot fabricate a finding or damage the evidence. It drives a read-only tool surface over a deterministic, signed engine, and a verdict guard re-derives every reported claim from the signed leaf rather than from the model's prose.

How we built it

The system is three thin layers over the existing, MIT-licensed logflip engine (NTFS $LogFile reverse-replay timestomp detection), which is reused unmodified:

• sift-mcp is a FastMCP server that exposes the engine as six typed, read-only forensic tools (scan_image, detect_record, inspect_mft, inspect_usnjrnl, verify_leaf, verify_db). There is no write, delete, or shell tool in the surface, so the agent physically cannot mutate or spoliate the evidence.
• sift-agent is a tool-calling loop with interchangeable drivers: Claude (Anthropic tool-use API), OpenAI (chat-completions tool calling), and a deterministic analyst policy that needs no API key and is fully reproducible. All three share the same guards: a max-iterations cap, a verdict guard, and a structured JSONL session log.
• An audit layer where every finding's verdict links to the tool execution that derived it (produced_by_seq), a corroborated anomaly links the independent tools that corroborated it (corroborated_by_seq), and journaled findings link to a signed leaf.

This combines two of the hackathon's supported patterns: a Custom MCP Server (#2) and a Direct Agent Extension (#1). The "never-false-confirm" invariant requires all four engine gates plus a real engagement key plus a signed fingerprint DB, so the demo's honest ceiling is provisional. Evidence is HMAC-signed over RFC 8785 canonical JSON and re-verifiable offline, and the result schema is closed (Pydantic extra="forbid") so no verdict can be smuggled through. It was built test-first (42 tests). The demo video composites a Remotion motion layer over a real, captured terminal run.

Challenges we ran into

• Making self-correction honest. A single-source $SI-vs-$FN delta with no $LogFile coverage is exactly the shape of a raw-disk edit that bypassed the journal. The tempting bug is to "confirm evil." The correct behavior is to corroborate and then refuse to escalate, because the engine has no journal to cryptographically invert. The self-correction had to be toward honesty, not toward a louder verdict.
• Testing an autonomous loop deterministically. LLM behavior is not reproducible, so the orchestrator is client-agnostic. A deterministic policy and an injected fake model client let the loop's control logic, guards, and self-correction be unit-tested without a key.
• Resisting the urge to over-claim breadth. The engine is deep on NTFS and silent on memory and network. Rather than fake coverage, the scope is stated plainly in the docs.

Accomplishments that we're proud of

• The guarantees are architectural, not prompt-based. A misbehaving model still cannot run a destructive tool (none is registered), cannot produce a confirmed finding the leaf does not support (the verdict guard and engine gates reject it), and cannot loop forever (the iteration cap halts it). This is tested directly, including test_model_prose_cannot_forge_a_confirmed_verdict.
• Zero hallucinated findings on the demo case: 0 false positives, 0 false confirmations, 0 missed artifacts. The underlying engine ships 809 tests and a measured 0.000 false-positive rate; the agent adds 42.
• It always runs for a judge. With no API key the agent falls back to the deterministic policy, so the entire triage is reproducible offline.
• A signed, offline-verifiable evidence trail where every claim traces to a specific tool execution and an HMAC-signed leaf.

What we learned

• Architectural guardrails beat prompt guardrails. The most defensible parts of this submission are the ones a misbehaving model cannot break: a tool surface with no destructive capability, a closed result schema, and a verdict guard that reads the signed leaf. The prompt only decides ordering.
• The hardest forensic value was already deterministic. Wrapping a tested, signed engine and adding autonomy on top produced a stronger, more trustworthy result than trying to make an LLM do the forensics directly.

What's next for logflip-sift-agent

• Validate against real acquired NTFS images and the SANS starter case data.
• Ship a populated, signed fingerprint DB so the confirmed tier is reachable in a demo (today the honest ceiling is provisional).
• Add more read-only MCP corroboration tools (prefetch, registry LastWrite, event-log gaps), each typed and non-destructive.
• A cross-source correlation mode that pairs the disk timeline with a memory capture.

Built With

• anthropic-claude
• docker
• fastmcp
• hmac
• model-context-protocol
• ntfs
• openai
• pydantic
• pytest
• python
• rfc-8785