AgentSentinel-MCP

Autonomous multi-agent red-teaming for clinical AI — turns every MCP-native adversarial test run into EU AI Act (Art. 9/12/15) compliance evidence.

Comment 1

Inspiration

Clinical AI is shifting from single-model chatbots to multi-agent pipelines — a retrieval agent pulls patient context, a reasoning agent drafts a recommendation, a tool-calling agent writes to the EHR. Every handoff between those agents is a place safety can silently fail, yet the industry still tests these systems the way it tested single models: manually, inconsistently, and long before the EU AI Act's Article 9/12/15 obligations actually bite in production. Security matured by building CVE — a public, versioned, coordinated disclosure process. Clinical agent safety has nothing equivalent. We wanted to build that.

What it does

AgentSentinel-MCP is an autonomous swarm of adversarial agents that continuously red-teams any MCP-connected clinical AI system. Five specialist probes — prompt-injection, drug-interaction trap, PHI-leak, permission-escalation, and hallucination-bait — attack the target concurrently through the Model Context Protocol, and every finding is automatically severity-scored and mapped to the specific EU AI Act article it constitutes evidence for (Article 9 risk management, Article 12 record-keeping, Article 15 accuracy/robustness). The result streams into a public, CVE-style Disclosure Ledger, so compliance evidence is a byproduct of testing, not a PDF written after the fact.

How we built it

We designed the orchestrator around LangGraph, using a shared blackboard state so probe agents can coordinate findings rather than run in isolation. The MCP layer lets the orchestrator discover a target's exposed tools and resources dynamically instead of hardcoding integrations per system. Severity scoring is rules-based rather than purely LLM judgment, grounded via RAG over clinical harm taxonomies, so results stay reproducible run to run. On the frontend, we built a multi-page Next.js site with an interactive demo that actually runs a simulated scan end-to-end — agents dispatching live, findings streaming into the ledger, and a real downloadable JSON compliance report — so the product proves itself instead of just describing itself.

Challenges we ran into

Distinguishing a genuine safety failure from a target system correctly refusing an unsafe request required building a verification step that re-checks findings before they're logged, to keep false positives down. Keeping LLM-based severity judgments reproducible meant constraining every score with structured output schemas rather than trusting free-text reasoning. And making sure the orchestrator's MCP attack-surface discovery could never trigger a genuinely destructive tool call meant defaulting hard to sandboxed targets, with live-system testing as an explicit opt-in only.

Accomplishments that we're proud of

Getting the Disclosure Ledger to feel like a real, live compliance artifact rather than a marketing widget — severity, timestamp, and article mapping all flow from one consistent data model across the homepage, the full disclosures table, and the interactive demo. We're also proud of building a demo page that's genuinely functional: it runs a real simulated scan and generates a real downloadable report, not a static screenshot pretending to be a product.

What we learned

That multi-agent clinical AI failure modes are fundamentally different from single-model failure modes — the interesting bugs live at the handoffs between agents, not inside any one model's output. We also learned how much clearer a compliance story becomes when it's structural (every finding auto-mapped to an article) rather than narrative (a human writing a report explaining why the system is compliant).

What's next for AgentSentinel-MCP

Packaging it as a one-line GitHub Action so any team can drop continuous red-teaming into CI on every PR that touches an agent pipeline; adding a responsible-disclosure/embargo workflow before findings on real deployed systems go public, mirroring coordinated disclosure in security; and extending MCP compatibility shims to cover the growing set of clinical agent frameworks beyond the reference implementations we tested against.

Inspiration

Clinical AI is shifting from single-model chatbots to multi-agent pipelines — a retrieval agent pulls patient context, a reasoning agent drafts a recommendation, a tool-calling agent writes to the EHR. Every handoff between those agents is a place safety can silently fail, yet the industry still tests these systems the way it tested single models: manually, inconsistently, and long before the EU AI Act's Article 9/12/15 obligations actually bite in production. Security matured by building CVE — a public, versioned, coordinated disclosure process. Clinical agent safety has nothing equivalent. We wanted to build that.

What it does

AgentSentinel-MCP is an autonomous swarm of adversarial agents that continuously red-teams any MCP-connected clinical AI system. Five specialist probes — prompt-injection, drug-interaction trap, PHI-leak, permission-escalation, and hallucination-bait — attack the target concurrently through the Model Context Protocol, and every finding is automatically severity-scored and mapped to the specific EU AI Act article it constitutes evidence for (Article 9 risk management, Article 12 record-keeping, Article 15 accuracy/robustness). The result streams into a public, CVE-style Disclosure Ledger, so compliance evidence is a byproduct of testing, not a PDF written after the fact.

How we built it

We designed the orchestrator around LangGraph, using a shared blackboard state so probe agents can coordinate findings rather than run in isolation. The MCP layer lets the orchestrator discover a target's exposed tools and resources dynamically instead of hardcoding integrations per system. Severity scoring is rules-based rather than purely LLM judgment, grounded via RAG over clinical harm taxonomies, so results stay reproducible run to run. On the frontend, we built a multi-page Next.js site with an interactive demo that actually runs a simulated scan end-to-end — agents dispatching live, findings streaming into the ledger, and a real downloadable JSON compliance report — so the product proves itself instead of just describing itself.

Challenges we ran into

Distinguishing a genuine safety failure from a target system correctly refusing an unsafe request required building a verification step that re-checks findings before they're logged, to keep false positives down. Keeping LLM-based severity judgments reproducible meant constraining every score with structured output schemas rather than trusting free-text reasoning. And making sure the orchestrator's MCP attack-surface discovery could never trigger a genuinely destructive tool call meant defaulting hard to sandboxed targets, with live-system testing as an explicit opt-in only.

Accomplishments that we're proud of

Getting the Disclosure Ledger to feel like a real, live compliance artifact rather than a marketing widget — severity, timestamp, and article mapping all flow from one consistent data model across the homepage, the full disclosures table, and the interactive demo. We're also proud of building a demo page that's genuinely functional: it runs a real simulated scan and generates a real downloadable report, not a static screenshot pretending to be a product.

What we learned

That multi-agent clinical AI failure modes are fundamentally different from single-model failure modes — the interesting bugs live at the handoffs between agents, not inside any one model's output. We also learned how much clearer a compliance story becomes when it's structural (every finding auto-mapped to an article) rather than narrative (a human writing a report explaining why the system is compliant).

What's next for AgentSentinel-MCP

Packaging it as a one-line GitHub Action so any team can drop continuous red-teaming into CI on every PR that touches an agent pipeline; adding a responsible-disclosure/embargo workflow before findings on real deployed systems go public, mirroring coordinated disclosure in security; and extending MCP compatibility shims to cover the growing set of clinical agent frameworks beyond the reference implementations we tested against.

Built With

  • docker
  • fastapi
  • fhir/hl7
  • framer-motion
  • github-actions
  • hugging-face-transformers
  • langgraph
  • litellm
  • model-context-protocol-(mcp)
  • next.js
  • opentelemetry
  • postgresql
  • pydantic
  • python
  • tailwind-css
  • typescript
  • vercel
Share this project:

Updates

Submission history