Sentinel Loop

Sentinel Loop console showing red-team co-evolution, with detection rising from 0.786 to 0.996.
Per-vector detection across rounds A-01 to D-01 with Round 2 mutation logs and capping stats
Tactic mutation logs showing feature perturbations like inter_event_seconds to test model limits.
Attack Atlas listing threat vectors across categories like graph topology and agentic authorization
Fidelity Scorecard evaluating 400k synthetic transactions against the PaySim anchor dataset.
Scatter plot mapping Amount KS vs Discriminator AUC to verify synthetic traffic realism
Defense Dashboard tracking 14 attack vectors with Recall @ 1% FPR, precision, and F1-scores.
Aggregate accuracy metrics (0.993 Recall) paired with decision latency across L1, L2, and L3 layers.
Live simulation feed categorizing transactions into Allow, Step Up, Hold, and Block bands
Case analysis of a Block decision (Risk: 0.993) with layer contributions from L1, L2, and L3.

Inspiration

The Widening Fraud Gap: Generative AI has dramatically lowered the cost of orchestrating complex cyberattacks—such as deepfake audio/video calls, synthetic identity theft, and tailored scams—costing bad actors mere cents to execute at scale.
The Blind Spot of Traditional Models: Standard fraud detection systems are inherently reactive; they rely on historical labeled data and can only recognize fraud that has already occurred, leaving real customers to bear the cost of first-wave attacks.
Emerging Threat Surfaces: As payments move toward agentic checkouts and LLM-driven interactions, the AI stack itself becomes an uncatalogued target for prompt injections, agent-in-the-middle substitutions, and model probing.

Closed-Loop AI Red-Teaming: Sentinel invents prospective GenAI attack vectors, simulates them safely on synthetic datasets, and trains defensive models before threats hit production networks.
Comprehensive Vector Taxonomy: Identifies 34 GenAI attack vectors, including 6 first-class AI-stack threat vectors (e.g., prompt injection in agentic checkouts, tool-schema abuse, model probing) absent from conventional fraud taxonomies.
Explainable Multi-Layer Scoring: Combines Rules, Graph + Gradient Boosted Machines (GBM), and a Semantic Layer to produce a calibrated risk score accompanied by ranked, plain-language reason codes.

Four-Stage Pipeline: Built on an Identify $\rightarrow$ Generate $\rightarrow$ Defend $\rightarrow$ Arena architecture.
Synthetic Simulation Network: Created a 3-layer simulator (narrative, mechanics, stats) running over a 400,000-transaction synthetic payment network.
Adversarial Arena (Co-evolution): Implemented an automated loop where the attacker mutates parameters against the deployed defender model, and harvested evasion data is fed back into retraining iterations.
Low-Latency Inference Pipeline: Optimized response pipelines to execute decisions within a 10.6 ms budget per 1,000 decisions to meet real-time authorization constraints.

Uncharted Attack Vectors: Cataloging and modeling attack vectors targeted directly at the AI stack itself (such as delegated-authority scope escalation) without existing industry taxonomies or historical training data.
Real-Time Latency Budgets: Balancing multi-layered detection—specifically the computation-heavy Graph + GBM layer (which takes ~9.88 ms)—to stay well within strict live authorization limits.
Safety & Privacy Constraints: Ensuring synthetic attack data accurately mirrored real-world signal footprints (e.g., carrier mismatches, velocity bursts) without generating actual deepfakes, malicious payloads, or exposing sensitive user data.

Co-Evolutionary Performance: Raised detection rates from 0.786 to 0.996 across 5 arena rounds while the adversary was actively modifying attack strategies.
High Operational Accuracy: Achieved a 0.993 Recall @ 1% False Positive Rate (FPR) on synthetic validation sets.
Statistical Data Fidelity: Validated synthetic transaction fidelity against 6.36 million real transaction rows with a Kolmogorov-Smirnov (KS) distance of 0.0718.
Transparent Decisions: Delivered fully explainable outputs with ranked reason codes in 10.6 ms per 1,000 decisions.

Focusing on Signal Footprints: Complex attacks are best identified by analyzing transaction signal chains (e.g., payee addition shortly before large transfers) rather than waiting for explicit historical labels.
Proactive Defense via Red-Teaming: Continuously probing a defense model against automated adversarial mutations is far more effective and ethical than static model updates or periodic manual audits.

Expanding Coverage: Scaling detection coverage across all 34 catalogued GenAI vectors beyond the initial active set.
Fintech & Bank Pilots: Partnering with payment service providers (PSPs), UPI applications, and agentic-commerce platforms to pilot test defenses in sandboxed environments.
Standardizing Industry Benchmarks: Working alongside financial regulators (such as RBI and NPCI) and card networks to establish a standardized, testable benchmark for GenAI fraud readiness.

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.