Heimdall Cyber
Building the strongest cyber and fintech defence from LLM reasoning and agentic AI — catching the attacks ordinary defences miss, where AI reasons through the **whole lifecycle of the system, not just one component.
The thesis in one line: rules decide, the LLM reasons only in the gray zone, a human owns the escalations — and the hardest attacks are caught by computing the right signal, not by a smarter model.
This is the full story of Heimdall Cyber, from first principles to the last experiment: what provoked it, what it is, how every piece works, how it was built on our prior system Discovery.AI, every pivot we made, every experiment we ran, what each result actually proves, and where it falls short.
Part I — What provoked this
Security almost always reduces to one quiet truth: an attack is never a single event, it is a campaign across a lifecycle. A real account takeover runs as a chain —
$$\textsf{credential stuffing} \rightarrow \textsf{takeover} \rightarrow \textsf{payout change} \rightarrow \textsf{card testing} \rightarrow \textsf{cash-out}$$
— and every step, viewed alone, looks ordinary. The defences the industry actually deploys each watch one layer and forget between layers:
- Per-IP bot management / rate limiting watches the network layer. It is defeated by residential-proxy rotation. One documented attack used 800,000+ residential IPs in ~6 hours, ~20 tries per IP then rotate — per-IP volume never crosses a threshold.
- Risk-based auth (RBA) scores a single login and does not carry suspicion forward into the money layer.
- SIEMs correlate security logs but do not natively join the transaction lifecycle an attacker actually walks.
Nobody joins the campaign across the lifecycle. That join is Heimdall's entire reason to exist. The single hardest instance — the one this project set out to defeat — is the distributed, low-and-slow campaign: a fleet of bots, each on a distinct residential IP, each staying under every per-IP threshold, coordinated on a shared set of target usernames. It is invisible per-IP and, until this work, invisible in aggregate.
We also read the field honestly before building. The 2025–26 agentic-security surveys describe an autonomy gap: of 76 agentic-security systems, 88% of offensive ones reach the top autonomy level and zero of 34 defensive systems do. The loudest finding across SOC literature is that the field is starving for reproducible empirics, not for model capability — the standard defensive benchmark (CAGE/CybORG) has reproducibility bugs, and surveys note "few empirical studies exist." That told us exactly where a contribution could land: a reproducible, application-layer, fraud-lifecycle benchmark with ablations.
Part II — Lineage: why we built on Discovery.AI
Heimdall is not greenfield. It is built on Discovery.AI, our own agentic knowledge-graph investigation system, and the decision to build on it was deliberate.
What Discovery.AI is
Ask Discovery.AI a question in plain language and it investigates recursively — an agent that decides, one step at a time, whether to answer, sharpen into a sub-question, or admit a real boundary — and builds a persistent, typed knowledge graph (Neo4j) of real entities, typed relationships and retrieved evidence, each with an honest confidence score. Its signature idea is a strict separation:
$$\textbf{World Model} \;\neq\; \textbf{View}$$
The graph is written once by investigation and never rebuilt when a user zooms or changes lens; views are reads with zero LLM calls. It is rendered with Cytoscape, and it was validated by a 10/10 synthetic-topology test suite fed straight into the renderer with no LLM in the loop — verified, not claimed.
The three things Heimdall inherited — and why they mattered
- The cardinal rule. Discovery's "views are reads, zero LLM calls" is the same discipline as Heimdall's "the verdict is LLM-free." We kept the principle that outlived everything else: the AI may explain a decision; it must never make one. This is the root of the entire tiered design.
- The frontend and the graph engine. The very Cytoscape + fcose engine that renders Discovery's knowledge graph now renders Heimdall's live attack graphs and system mind-maps — same theme, same WASM fluid background, same docs treatment. When Heimdall shows "the attacker, the move they tried, and the rule that caught it, wired together," that is Discovery's graph engine, repurposed.
- The methodology of honest verification. Discovery's "10/10 topology suite, no LLM in the loop" is the direct ancestor of Heimdall's ablation benchmarks: measure the thing, remove one ingredient, prove the contribution. Verified, not asserted, carried from a knowledge-graph context into a security one.
The pivot: v1 → Cyber
This is the biggest pivot in the project's history, and it is worth stating plainly.
- Heimdall v1 was a financial decision system — recovery, risk, and reconciliation — that called modules one decision at a time, with Discovery.AI embedded as its investigator (the component that explained a decision). It answered "what should we do about this transaction?"
- Heimdall Cyber is a security system — a control point that actions pass through, inline, lifecycle-wide. It answers "should this action happen at all, and what does the whole campaign around it look like?"
Same honesty discipline, same visual language, a different and harder problem. The one idea that survived the pivot intact — "the LLM explains, it never decides" — became the architectural spine of the new system.
Part III — Architecture
Heimdall is an inline guard (financial_system/cyber/guard.py) placed in front of an otherwise ordinary application (a storefront). The application is never modified. Heimdall reads only what a reverse proxy can see — the request, its source IP, the response — and enforces exactly one deterministic verdict per action. Turn the guard off and the same app is defenceless; turn it on and it holds. That A/B is the demo.
The pipeline for every sensitive action (login, account change, checkout, capture, payout, object read):
$$\textsf{request} \to \texttt{enrich()} \to \texttt{detectors} \to \texttt{playbooks.compose} \to {\textsf{Tier-1 verdict}} \to [\textsf{gray zone?} \to \textsf{Tier-2}] \to \textsf{act}$$
Four properties define it:
- Measurement is separate from the verdict.
enrich()computes signals — velocity counts, new-device, impossible-travel, IP classification, the cross-IP campaign aggregate. A signal is a fact; thresholds and decisions live only in the detectors. This separation is exactly what makes the system auditable and ablatable. Formally, let $S$ be the signal vector and $D$ the decision; we enforce $S \perp!!!\perp \text{(thresholds)}$ — the enricher never sees a threshold, the detector never computes a raw signal. - The decision is deterministic and cited.
playbooks.composetakes the fired alerts and returns the most-restrictive decision with the singlerule_idthat produced it. "Why was this held?" always has a literal answer. No LLM is in this path. - Cross-layer memory is the system. When a detector fires it writes a flag (
watch,hold) onto the named entity inCyberState; a later action reads those carried flags. This is the join RBA / bots / SIEM never make — an account-takeoverwatchat the login layer is what makes the cash-out detector hold the money. - Reversibility is respected. Logins and authorizations are observed (reversible). The irreversible step — capture — is decided before it runs, and a block voids the held authorization, so the money is released, not taken.
Part IV — Detection: the catalog and the contribution
Each detector is a deterministic check with stated modeling-assumption thresholds (the knobs a purple-team cycle tunes), mapped to an ATT&CK / F3 kill-chain stage, returning an alert or nothing.
| Rule | Stage | Fires on | Verdict |
|---|---|---|---|
| CS1 credential stuffing | Initial Access | ≥10 distinct usernames and ≥5 failures from one IP / 15 min | CHALLENGE |
| CAMPAIGN1 (contribution) | Initial Access | ≥6 distinct IPs failing one username × ≥3 such usernames | CHALLENGE |
| BF1 brute force | Initial Access | ≥8 failures from one IP | CHALLENGE |
| TOR1 | Initial Access | login over a Tor exit | CHALLENGE |
| BEHAV1 navigation coherence (contribution; gateway only) | Initial Access | a login whose client behaves like a bot — no page/asset loads, machine timing, UA churn (coherence < 0.25) | CHALLENGE |
| ATO1 account takeover | Positioning | success on a known account, new device and (datacenter/Tor ∨ impossible travel ∨ breached pwd) | CHALLENGE + tag watch |
| PC1 payout change | Positioning | payout/email change on a watch/hold/suspect account |
HOLD |
| SES1 session hijack | Positioning | a session used from a new device and a datacenter network | HOLD |
| BOLA1 IDOR | Collection | a session reads an object it does not own | DENY |
| CARDTEST1 | Execution | ≥4/5 card-testing indicators on a device | DENY |
| MON1 cash-out | Monetization | money movement on a watch/hold account |
HOLD + void authorization |
The gap we found, and CAMPAIGN1
Every rule above except CAMPAIGN1 existed before this work. CS1 keys only on per-IP velocity. We audited every signal enrich() computed and every CyberState query, and found no cross-IP aggregate and no username-keyed lookup anywhere in the engine. Consequently a fleet of $K$ bots, each on a fresh residential IP, each below CS1's thresholds, hitting a shared username set, was invisible per-IP and in aggregate — nothing joined the attempts.
CAMPAIGN1 is the exact transpose of CS1. Where CS1 counts distinct usernames per IP, CAMPAIGN1 counts distinct IPs failing one username, plus a multi-username corroboration. Let $F$ be the set of failed-login edges $(\text{ip}, \text{user})$ in the window. Then:
$$\text{CS1 keys on } \max_{\text{ip}} \big|{\, u : (\text{ip}, u) \in F \,}\big| \qquad\text{(usernames per IP)}$$
$$\text{CAMPAIGN1 keys on } \big|{\, u : |{\text{ip} : (\text{ip}, u) \in F}| \ge \tau_{\text{ip}} \,}\big| \ge \tau_{\text{user}} \qquad\text{(IPs per username, across a set)}$$
The insight that makes it robust: a fleet can rotate its IP, device, user-agent and password, but it cannot rotate the shared target set — that is intrinsic to being a campaign. The detector keys on that structural invariant, backed by a bounded, time-windowed state of non-success logins only (fails/blocked — the first false-positive guard, since legitimate traffic succeeds). It writes the same watch tag ATO1 writes, so the existing cash-out chain holds the money with no new downstream plumbing.
Part V — The reasoning core: tiered, not blended
The requirement was to give the rules and an LLM a voice, plus a human in the loop. The obvious design —
$$\text{final} = \alpha \cdot \text{rule} + \beta \cdot \text{llm} \;\xrightarrow{}\; \text{threshold}$$
— was rejected, for three reasons grounded in the literature: (1) it destroys auditability ("blocked because the sum was $0.57$"); (2) it makes the money-gate non-deterministic, so it can no longer be cleanly ablated or benchmarked; (3) it lets a confident-but-wrong model dilute a hard rule. The security literature is explicit that LLM output must not flow straight into a blocking decision without a deterministic gate.
What we built is tiered and routed by confidence, not blended:
- Tier 1 — deterministic rules = the authoritative floor. Clear attack → DENY/HOLD (final); clean → ALLOW (final). The LLM never overrides a block and never authorizes money.
- Tier 2 — the reasoner carries weight only in the gray zone Tier 1 could not resolve. It is raise-only (can tighten, at most to HOLD — never issues a hard DENY itself), confidence-gated, and may never produce an ALLOW of money the rules did not already allow. Formally, if $v_1$ is the Tier-1 verdict and $v_2$ the reasoner's, the composed verdict is
$$v = \max\big(v_1,\; v_2 \cdot \mathbb{1}[\,c \ge c_{\min}\,]\big), \qquad v_2 \le \textsf{HOLD}$$
where $\max$ is over the restrictiveness order $\textsf{ALLOW} < \textsf{CHALLENGE} < \textsf{HOLD} < \textsf{DENY}$ and $c$ is calibrated confidence. The default reasoner is a deterministic heuristic (so the hybrid stays reproducible); a real LLM is opt-in behind the same interface.
- Tier 3 — a human cyber-engineer adjudicates whatever Tier-2 escalates (live console at
/tier3). The action is already safe while it waits (Tier-2 only ever raised it). The human's decision becomes labeled data — the signal that calibrates Tier-2 and can be promoted into a new deterministic Tier-1 rule. That is the learning loop, without surrendering determinism.
Crucially, Tier-2 is opt-in (decide(…, reasoner=…)). With no reasoner, behaviour is pure Tier-1, so the deterministic baseline — and every existing test and benchmark — is byte-for-byte unchanged, and the hybrid is measured as an added arm, never a confound.
Part VI — Methodology: why the evidence is trustworthy
Inherited from Discovery.AI's "verified, not claimed," sharpened for security:
- Ablation is the method. "Heimdall saved \$900" is meaningless alone — the attack might have been weak. Every benchmark runs the same attack with the guard fully on, fully off, and with one ingredient removed at a time. If removing an ingredient lets the attack through, that ingredient did real work; if it doesn't, it's decoration. Every claim is therefore falsifiable.
- No future-information leakage. Decisions read only events at or before "now"; the event is recorded after the decision, so a challenged login never becomes a trusted success.
- Deterministic, seeded, offline. The attack range is synthetic and reproducible. "Residential IPs" are strings classified by a bundled net-intel table of the same shape as production lists. No real botnet, no VPN abuse, no attack on any third-party site.
- Decoys are first-class. Every run reports false-positive behaviour on benign look-alikes (a traveller, a VPN user on their own device, a corporate-NAT office, a popular shared login) — not just the catch rate. A defence is only as good as its friction is low.
Part VII — Every experiment, in depth
Four facets run reproducibly today (phase1_benchmark.py runs them all).
Facet 1 — Lifecycle detection + ablation → "it is a system, not a bag of detectors"
Runs the full F3 campaign (stuffing → takeover → payout change → card testing → cash-out) in configurations: no-guard, full guard, and each ingredient removed. Result: full guard = \$0 stolen, no email hijack, ring detected; every ablation regresses — most tellingly, −cross-layer (remove the carried-flag memory) lets the email hijack succeed again. Deeper meaning: the value is not any single detector, it is the carried watch tag flowing from the login layer into the money gate. Removing the join — not a detector — is what re-opens the attack. This is the measured proof of the central thesis.
Facet 3 — Distributed low-and-slow, A/B/C → "you cannot reason your way out of a missing signal"
The distributed fleet (8 bots, 8 residential IPs, each under CS1) with a cracked credential cashed out from a same-region residential session, through the same guarded storefront, in three arms:
| Arm | Money through | CAMPAIGN1 | CS1 (per-IP) |
|---|---|---|---|
| A no-guard | \$900 | — | never |
| B cross-IP signal ablated | \$900 | False | never |
| C full | \$0 | True (8 challenges) | never |
All four decoys clean. Arm B is the philosophical core. It is the honest stand-in for "keep the old system and just prompt the LLM harder / give it more examples" — and it misses identically to no-guard. Because the decision path is LLM-free and a detector can only act on a signal enrich() computes, and the cross-IP aggregate did not exist, no amount of model cleverness could conjure it. The fix had to be infrastructure (a new computed signal), not intelligence. And CS1 never firing in any arm is the concrete death certificate for per-IP rate limiting against rotating residential proxies.
Facet 5 — Adaptive-evasion robustness → "robust by invariant, not by threshold"
An adaptive attacker that does not guess the limit but measures it: a scout ramps failed logins from one probe IP until Heimdall first challenges, learns the limit $L$, then holds the whole fleet one step under it. Result: scout discovered $L = 8$ (it tripped BF1), the fleet held at $7$/IP — provably under every per-IP rule — and \$0 was stolen; CAMPAIGN1 still caught it. Deeper meaning: a static-threshold defence dies to this (Goodhart's law — once a threshold is a target, the adversary optimizes against it). CAMPAIGN1 survives because it keys on a structural invariant, not a volume threshold. The attacker can tune its per-IP rate to anything, but it cannot stop being "a set of usernames failing from many IPs" without ceasing to be a campaign. That is the difference between a defence evadable by parameter tuning and one evadable only by abandoning the attack goal — the latter is the gold standard.
Facet H — Tier-2 hybrid → "the LLM where its upside is highest and its downside bounded"
A labeled set of gray-zone cases Tier-1 misses (an account flagged suspect moving money from a new device — MON1 gates watch/hold, not suspect), and a set of clean look-alikes, through decide() with Tier-2 off vs on. Result: gray-zone caught Tier-1 0/3 → Tier-1+2 3/3, with 0 false positives on 4 clean cases. Deeper meaning: deterministic rules enumerate known shapes — precise but brittle at the boundaries no one enumerated. The LLM's value is generalization to those un-enumerated combinations; the architecture is why it is safe — Tier-2 only engages in the gray zone, only raises, cannot touch the money gate, and benign decoys never even reach it. The triage literature (Alert-Bench) found that more LLM compute often makes triage worse through false positives; the 0-FP number is the measured proof that confining the probabilistic component to the uncertain band captures its generalization without its unreliability.
Results, summarized
| Claim | Proven by | Number |
|---|---|---|
| It is a lifecycle system, not point-detectors | Facet 1 ablation | full guard \$0; −cross-layer re-opens the hijack |
| It catches what per-IP cannot | Facet 3 A/B/C | A \$900 · B \$900 · C \$0; CS1 never fires |
| A prompt cannot substitute for a missing signal | Facet 3 arm B | B misses identically to no-guard |
| Robust to an adaptive adversary | Facet 5 | scout finds $L=8$, fleet holds at $7$, \$0 stolen, caught |
| The LLM layer adds catch at zero FP cost | Facet H | Tier-1 0/3 → Tier-1+2 3/3; 0/4 false positives |
| Benign traffic is untouched | all facets | NAT, VPN, traveller, popular-username decoys all clean |
Every adjective in the thesis — neuro-symbolic, lifecycle-wide, adaptive-robust, human-governed — is one ablation away from being disproven, and none are.
Part VIII — Every pivot and change (the build journal)
This is the honest sequence of how the system actually came to be:
- The v1 → Cyber pivot. We stopped building a financial decision system (recovery/risk/reconciliation that calls Discovery.AI) and built a security control point that actions pass through. The surviving idea: LLM explains, never decides.
- The gap audit. Before writing a line of new detection, we audited every signal and every state query in the existing engine and proved, concretely, that no cross-IP / username-keyed signal existed — so the distributed campaign was provably invisible. This made the contribution precise instead of speculative.
- Building CAMPAIGN1 (stories S1–S6). Windowed cross-IP state (
events.py), the enrich aggregate (distinct_ips_failing_user_15m,campaign_targets_15m,campaign_ip_pool_15m), the detector itself (the transpose of CS1), the ablation switch, and a distributed attack generator with cross-IP decoys and benign seed accounts. - A confound we found and fixed. In the first A/B/C run, arm B (cross-IP ablated) was saving the money — but for the wrong reason: an unrelated session rule (SES1) was quietly catching the cash-out. We fixed the experiment so the cash-out uses the residential win identity in the victim's own region, removing the confound. Only then did arm B honestly miss identically to no-guard. This is the moment the proof became trustworthy — and it is exactly the kind of thing that ablation discipline is for.
- The tiered-reasoning refactor. We rejected the weighted-blend design and built
reasoner.py(Tier-2 gray-zone, raise-only, confidence-gated) +escalation.py(Tier-3 queue + adjudication), opt-in so the deterministic baseline stayed byte-for-byte unchanged. - The adaptive attacker (S7). We added a scout that measures the per-IP limit and holds the fleet under it, to attack our own defence at its weakest assumption — and showed CAMPAIGN1 survives because it is invariant-keyed.
- Phase-1 benchmarks. Consolidated all four facets into one reproducible run; every facet passes.
- The Tier-3 human console. A judge-facing operations console (
/tier3) that runs the Phase-1 proof live and shows the human-in-the-loop escalation queue, where a "Confirm block" turns a decision into labeled data. - Generalising past our own app. A reverse-proxy gateway (
cyber/proxy.py) puts Heimdall in front of any upstream app, observing the whole request stream — which let us add BEHAV1 (navigation-coherence: a login whose client never loads a page or asset, with machine timing and UA churn, behaves like a bot). - A real third-party app (Chirp). We stood up a real Node social app Heimdall was never built for, and defended it through the reverse proxy with no app changes — the distributed campaign is caught the same way, and a takeover there is shown weaponising the account (posting scams from the victim's handle), to make the stakes concrete.
- The website, rebuilt on Discovery's frontend. We reused Discovery.AI's actual frontend — theme, graph engine, docs treatment — for Heimdall's landing, research docs, and the live workspace, stripping Discovery's own secrets and chat. The attack graph you see is Discovery's knowledge-graph engine repurposed.
- The "you've been hacked" reveal. A deliberately visceral demo surface: run the attack with the guard off and a full-screen reveal shows the cracked password, the silently changed email, the drained wallet — and, framed honestly as a teaching cheat, the extracted plaintext password with the "don't store plaintext" lesson — then a green "protected" variant when the guard is on.
Part IX — Where it sits in the literature
The 2025–26 surveys frame the autonomy gap (zero of 34 defensive agentic systems reach top autonomy) and a field starving for empirics. Heimdall's contribution lands precisely there — a reproducible, application-layer, fraud-lifecycle benchmark with ablations — and its design (neuro-symbolic, lifecycle-wide, provenance-first, human-governed) is more principled than many deployed "AI SOC" tools, which route model output straight into blocking decisions.
Part X — Limitations & threats to validity
We state these rather than hide them:
- A true viral / password-reset storm could resemble a distributed campaign (many real users failing from many IPs against a set of usernames). The fails-only edges + multi-target AND cover the common cases; the named mitigation — require the attacking IP pool be reused across targets (same bots, not a crowd) — is designed but not yet implemented.
- Thresholds are lab-scale (CAMPAIGN1: 6 IPs / 3 targets), tuned for demo legibility; production would run higher and add the pool-reuse guard.
- Synthetic range. The net-intel tables and attack traffic are bundled samples of the same shape as production lists, not production data. External-dataset validation (IEEE-CIS transaction fraud, CORTEX/GUIDE triage) is future work — it adds external validity, not a change to the mechanism.
- The deterministic default reasoner stands in for an LLM in the reproducible benchmark; a real LLM's gray-zone behaviour is bounded by the same raise-only / confidence-gated contract but not yet measured at scale.
Part XI — What we learned
- The hardest attacks are beaten by computing the right signal, not by a smarter model. A distributed campaign is invisible per-address and obvious the instant you join attempts across the pool. The whole problem is where you look.
- The bottleneck in applied AI security is often observability, not model capability. Arm B proves it: with the signal missing, no prompt rescues you.
- The right role for an LLM in a defence is narrow and powerful — reason about the ambiguous, never pull the trigger. Keeping the AI off the authoritative path is what makes the whole system auditable.
- A security claim is only worth its ablation. "It worked" means nothing until you show which ingredient made it work.
Part XII — Challenges we ran into
- Proving necessity, not just success — building A/B/C experiments that isolate the one primitive, and hunting down the SES1 confound that was hiding whether CAMPAIGN1 actually mattered.
- Catching the campaign without punishing real users — getting to zero false positives on travellers, VPN users and corporate-NAT offices, reported alongside every detection number.
- Standing up to an adaptive attacker — one that measures the threshold and stays under it, forcing the defence to key on an invariant rather than a volume.
- Guarding an app we don't control — reading only what a reverse proxy can see, changing none of the protected app's code, which forced the clean split between measuring signals and making decisions.
Part XIII — Demonstrated on PayPal
We proved it on a real payment rail: Heimdall authorizes through PayPal and, the moment a cash-out looks fraudulent, voids the authorization before capture — the money is released, not taken. The guard is a drop-in layer in front of an ordinary payments app; turn it off and the app falls, turn it on and it holds. That A/B is the live demo.
Part XIV — What's next
- Close the learning loop: promote confirmed Tier-3 adjudications into deterministic Tier-1 rules.
- External-dataset validation: IEEE-CIS transaction fraud and CORTEX/GUIDE triage.
- A deterministic entity-graph correlation layer to generalise CAMPAIGN1 beyond the hand-written transpose — without a black-box GNN, preserving auditability.
- The pool-reuse third factor for the viral-event false-positive case.
The north star is the strongest, most honest cyber and fintech defence we can build — one that generalises past any single app or rail. And as autonomous AI agents begin to act and transact on our behalf, the same engine becomes the guardrail that keeps a hijacked or manipulated agent from doing harm: rules decide, the AI reasons, a human owns what it isn't sure of — and the AI never acts alone.
References
Agentic-security survey (autonomy gap) — arXiv 2510.06445 · AI-augmented SOC maturity — MDPI JCP 5/4/95 · Banking AI security agent (anti-silo) — arXiv 2606.17555 · Neuro-symbolic survey — arXiv 2509.06921 · "LLMs discharge safety obligations too early" — arXiv 2607.03963 · LLM-proposes-rules (MoCQ) — arXiv 2504.16057 · CybORG++/CAGE reproducibility — arXiv 2410.16324 · Broken Gates (bot defences vs LLM agents) — arXiv 2607.18659 · FinFraudBench — arXiv 2608.15177 · CORTEX triage — arXiv 2510.00311 · cost-calibrated escalation — arXiv 2601.07006 · agentic-AI cybersecurity risks — arXiv 2601.05293. Built on Discovery.AI.