Why this exists
Most "malware" alerts in production are false positives. A binary in C:\Windows\Temp triggered by a scheduled task with obfuscated strings and a base64 PowerShell wrapper LOOKS exactly like an attack — and it is also exactly what Microsoft Endpoint Manager, IT scheduled maintenance scripts, and most third-party patch-management tools look like in their normal mode. A Tier-2 analyst who reads only the surface evidence pages the on-call, wakes up half the team, and writes "malware confirmed" in the runbook. Six hours later somebody from IT closes the ticket: "that was us, it ran under change CR-XXXX." Multiply that by 50 alerts a week and the team stops looking at the alerts at all.
protocol-sift-agent is what we want the Tier-2 analyst to actually be: form a working hypothesis fast, keep walking the evidence, and revise the verdict loudly in the open when the deeper data contradicts the surface read.
What it does
Takes a SIFT case ID and a question ("is this an incident?"), then walks seven Protocol SIFT MCP tools in order:
list_artifacts— what's attached to the caseexamine_binary— surface indicators (run from temp, scheduled-task triggered, obfuscated strings)compute_hash— SHA256 + digital signature chaincheck_ioc— four-feed lookup (Mandiant, CrowdStrike, AbuseDB, internal TI)get_process_tree— parent, grandparent, user contextquery_event_log— logon record + approved-change-management windowlist_persistence_mechanisms— sanctioned scheduled tasks vs org baseline
After steps 1–2 it MUST state an INITIAL HYPOTHESIS. After step 7 it MUST state a FINAL VERDICT. If those disagree, it MUST emit a labeled SELF-CORRECTION paragraph naming the exact pieces of evidence that flipped the verdict. The system prompt enforces this output contract; the test suite locks in the canned case so the flip is reproducible.
The canned case
CASE-2026-0518-001. update_helper.exe in C:\Windows\Temp. Surface evidence says malware. Walking all seven tools reveals: signed by Microsoft Corporation, IOC-clean across four feeds, parent process is DOMAIN\svc_endpoint_admin (sanctioned service account), event log shows a service logon from the Endpoint Manager network range during approved change CR-2026-MAY-0517, and the NightlyMaintenance scheduled task is 412 days old and matches the org baseline. The agent flips from MALWARE CONFIRMED to FALSE POSITIVE — sanctioned admin activity, and emits the SELF-CORRECTION paragraph explaining the flip.
How we built it
- Gemini 2.5 Flash on Vertex AI
google.adk.agents.LlmAgent+McpToolset(Google Cloud Agent Builder / ADK)- Protocol SIFT MCP stub written against the same tool surface as the real SIFT MCP server; the stub is what the agent talks to in the demo, the real server is one env var away
- Streamlit dashboard, Cloud Run deployable
- Apache 2.0, standalone repo created during the FIND EVIL contest window
Challenges we ran into
Writing a system prompt that drives genuine self-correction is harder than it sounds. Early prompt drafts had Gemini hedge ("the evidence is mixed, recommend further investigation"). The fix was to require labeled sections (INITIAL HYPOTHESIS / SELF-CORRECTION / FINAL VERDICT) with explicit text contracts: SELF-CORRECTION must name the contradicting evidence by name, FINAL VERDICT must be one of three strings, and identifiers must be quoted verbatim from tool output. The smoke test now checks that the agent calls update_helper.exe "Microsoft-signed," names svc_endpoint_admin verbatim, and quotes CR-2026-MAY-0517 exactly.
What we learned
A Tier-2 analyst's job is mostly about not stopping at the first piece of evidence. The same is true for an agent. Forcing a labeled checkpoint between "initial hypothesis" and "final verdict" is most of what makes the agent useful — without it Gemini happily writes confident verdicts off two tool calls.
What's next
Wire the agent against the real Protocol SIFT MCP server (env-var swap, the stub already mirrors the tool surface). Add a "second opinion" mode where two Gemini instances independently produce verdicts and a third agent reconciles them. Pipe the verdict trail into a SIEM as a structured event so the SELF-CORRECTION rationale ends up in the case file, not just the chat transcript.
Log in or sign up for Devpost to join the conversation.