Sentinel-Agent

An AI-driven, evidence-based penetration testing tool that deterministically detects and proves API authorization vulnerabilities (BOLA/IDOR).

Comment

💡 Inspiration

Broken Object Level Authorization (BOLA / IDOR) has reigned as the #1 vulnerability on the OWASP API Security Top 10 for years. Despite its prevalence, traditional Dynamic Application Security Testing (DAST) scanners like OWASP ZAP or Burp Suite fall short: they are inherently single-tenant and cannot understand object ownership or business logic—they simply don't know “who owns what.”

On the other hand, the emerging wave of "autonomous AI penetration testing agents" suffers from the opposite pathology: hallucinations and false positives. Naive LLM agents inspect HTML or API responses and prematurely claim a vulnerability exists based on vague pattern matches, flooding security engineers with unverified noise.

We asked ourselves: What if we combined the exploratory intelligence of LLMs with the unforgiving mathematical rigor of deterministic software verification?

This question gave birth to Sentinel-Agent: an AI-assisted, evidence-based access-control penetration testing framework where the LLM acts as the scout (hypothesizing attacks), and a deterministic code engine acts as the judge (proving vulnerabilities).

⚙️ How We Built It

Sentinel-Agent was built from the ground up using Python 3.11, designed around strict security invariants and a decoupled multi-actor architecture:

  1. Dual-Actor Differential Engine: The engine authenticates as two distinct actors ($A$ and $B$) and an unauthenticated actor ($\emptyset$), executing differential security tests across target API endpoints.
  2. Single Choke-Point Replayer (Replayer): The LLM never touches the network directly. It can only propose strongly typed, structured actions. Every outbound HTTP packet flows through a centralized choke point enforcing strict scope validation, IP pinning, rate limits, and zero secret leakage.
  3. Deterministic Oracle Architecture: Instead of relying on LLM vibes, specialized oracles (IdorOracle, BflaOracle, BoplaOracle, StateChangingOracle, etc.) analyze baseline vs. attack responses.

📐 Mathematical Formulation of the Verification Oracle

To guarantee zero false positives, an access-control vulnerability is labeled $\text{CONFIRMED}$ only when it satisfies formal multi-actor differential controls:

Let Actor $A$ own resource $R_A$ with unique leaked marker $M_A$, and let Actor $B$ be an attacker ($A \neq B$). Let $f(req, actor)$ denote an HTTP execution under the actor's authenticated session.

  1. Positive Control ($C_{\text{pos}}$): Legitimate owner $A$ can successfully access the resource: $$C_{\text{pos}} \iff \text{status}(f(\text{GET } R_A, A)) = 200 \land M_A \in \text{body}(f(\text{GET } R_A, A))$$

  2. Negative Control ($C_{\text{neg}}$): An unauthenticated actor ($\emptyset$) is denied access: $$C_{\text{neg}} \iff \text{status}(f(\text{GET } R_A, \emptyset)) \in {401, 403}$$

  3. Stability Invariant ($C_{\text{stable}}$): The baseline response remains invariant over successive probes: $$C_{\text{stable}} \iff \Delta(f_1(\text{GET } R_A, A), f_2(\text{GET } R_A, A)) \le \epsilon$$

  4. Verdict Ladder Function: $$\text{Verdict}(A, B, R_A) = \begin{cases} \mathbf{CONFIRMED}, & \text{if } C_{\text{pos}} \land C_{\text{neg}} \land C_{\text{stable}} \land \big(M_A \in \text{body}(f(\text{GET } R_A, B))\big) \ \mathbf{LIKELY}, & \text{if } C_{\text{pos}} \land \text{diff}(r_{\text{base}}, r_{\text{attack}}) \le \delta \land M_A \notin \text{body}(r_{\text{attack}}) \ \mathbf{REJECTED}, & \text{if } \text{status}(f(\text{GET } R_A, B)) \in {401, 403, 404} \ \mathbf{INCONCLUSIVE}, & \text{otherwise} \end{cases}$$

Every confirmed vulnerability outputs a reproducible repro_curl command, a diff view, and clear remediation instructions.

🚧 Challenges We Ran Into

  • Dynamic Entropy and Normalization: Real-world API responses are littered with volatile elements: timestamps, CSRF tokens, rotating session IDs, and UUIDs. Calculating diffs without false alerts required developing an entropy normalization pipeline that scrubs ephemeral values before comparison.
  • Session Cross-Contamination: Managing concurrent asynchronous HTTP sessions without leaking cookies or authorization tokens between Actor $A$ and Actor $B$ was critical. We enforced an invariant where each actor maintains an isolated httpx.AsyncClient session pool.
  • Safe State Mutation (BOLA-Write): Evaluating write and delete authorization (PUT/PATCH/DELETE) poses a risk of corrupting test databases. We designed a store-verify-revert state-machine oracle to confirm mutations on victim objects while preventing denial-of-service.
  • Prompt Injection & Scope Containment: In autonomous security tools, LLMs can inadvertently hallucinate out-of-scope targets or dangerous payloads. We solved this with non-bypassable, code-level pre-flight policy filters.

🏆 Accomplishments That We're Proud Of

  • 540+ Passing Unit & Integration Tests: A comprehensive, fully reproducible test suite testing every oracle and policy gate against mocked network traffic.
  • Zero-False-Positive Guarantee: Not a single CONFIRMED verdict is emitted without concrete proof (leaked markers and verified baseline controls).
  • Zero-Dependency Web Dashboard: Built an interactive web control panel and visualizer using Python's standard library (http.server), allowing users to run scans and inspect diffs with zero external frontend bloat.
  • Battle-Tested Calibration: Successfully identified and proved known complex BOLA/IDOR vulnerabilities in benchmark environments such as OWASP Juice Shop.

📚 What We Learned

  1. The Separation of Concerns in AI Systems: The most reliable agentic architectures treat LLMs as creative hypothesis generators, never as arbiters of truth or raw executors. Pairing AI reasoning with deterministic validation eliminates hallucination risks in high-stakes domains like cybersecurity.
  2. Deep Mechanics of API Access Control: Gained deep operational insights into how subtle architectural omissions in object-level permission models lead to catastrophic data breaches.

🚀 What's Next for Sentinel-Agent

  • GraphQL & gRPC Oracle Support: Expanding beyond REST to capture complex authorization graphs in modern microservices.
  • Automated Remediation PRs: Generating framework-specific code patches (FastAPI, Django, Spring Boot, Express) to fix verified vulnerabilities automatically.
  • CI/CD Integration: Packaging Sentinel-Agent as a GitHub Action / GitLab CI gate to block unauthorized API endpoints before they reach production.

Built With

Share this project:

Updates

Submission history