Inspiration
Security tools can discover vulnerabilities, but raw findings are difficult to trust and prioritize. A scanner may identify an exposed service, a weak configuration, and a leaked secret without explaining whether those findings form a realistic attack path.
HermesShield was inspired by the need for security automation that is both useful and reviewable. We designed it around a controlled pipeline:
$$ \mathrm{Scope} \rightarrow \mathrm{Tool\ Execution} \rightarrow \mathrm{Evidence} \rightarrow \mathrm{Correlation} \rightarrow \mathrm{Remediation} $$
The goal is not to give an AI agent unrestricted command access. The goal is to let Nemotron contribute security reasoning while HermesShield enforces the boundaries.
What It Does
HermesShield creates bounded security assessment plans, executes approved security skills, collects redacted evidence, and correlates findings into confidence-aware attack paths.
The project provides:
- Explicit target and action scope enforcement
- Allowlisted security tools with timeouts and environment restrictions
- Safe vulnerability validation without exploitation
- Secret and credential redaction before persistence
- Persistent memory states such as
VERIFIED,KNOWN, andHYPOTHESIS - Attack-path correlation with evidence and confidence tracking
- Remediation and assessment reports
- A browser-based interactive demonstration for judges
The demo lets judges follow the audit lifecycle from target scoping through model planning, boundary blocking, evidence collection, correlation, and report generation without using the command line or scanning a live target.
How We Built It
The core system is written in Python. HermesAgent coordinates scope validation, planning, execution, evidence collection, memory, and reporting.
Nemotron is accessed through Nebius Token Factory for bounded audit-plan generation. The model receives the target, objective, allowed skill names, and strict safety constraints. Its response must be structured JSON. HermesShield validates every returned skill against an allowlist, preserves plan dependencies, and prevents shell commands, credentials, arbitrary tools, and exploitation payloads from entering the plan.
The browser experience is powered by Flask and communicates with the backend through a JSON API. When Nebius credentials are unavailable, the interface clearly reports SAFE FALLBACK and runs a deterministic simulation so the demo remains reproducible.
Challenges We Ran Into
The main challenges were:
- Connecting model-generated planning to a deterministic execution engine.
- Preventing arbitrary model output from becoming an executable tool request.
- Preserving evidence while removing secrets and credentials.
- Correlating findings without overstating unsupported relationships.
- Turning a command-line security framework into a clear browser experience.
- Designing a demo that is interactive without requiring judges to configure scanners or access a live target.
We addressed these challenges by treating Nemotron as a bounded planning component rather than an unrestricted command executor, and by validating every transition between model output, tools, evidence, and reports.
Accomplishments That We're Proud Of
- Built an end-to-end browser demonstration instead of requiring judges to use the CLI.
- Added a real Nebius/Nemotron planning integration with structured output validation.
- Made model-generated skills influence the actual assessment plan while preserving dependencies.
- Implemented scope enforcement, tool allowlisting, timeout controls, and safe fallback behavior.
- Added evidence redaction and restricted evidence persistence.
- Built attack-path correlation that distinguishes observed facts, hypotheses, and validated relationships.
- Created a reproducible benchmark and adversarial security test suites.
What We Learned
We learned that an AI security system needs more than a capable model. It needs strong boundaries around model actions, clear separation between facts and hypotheses, evidence validation before reasoning, secret redaction before persistence, confidence tracking, and human-readable explanations.
We also learned that a strong security demo should show both successful behavior and blocked behavior. Demonstrating a rejected path-traversal request is more informative than showing only a successful scan.
What's Next for HermesShield
- Deploy the Flask demo to a public, judge-accessible URL.
- Add a richer live execution view for approved benchmark targets.
- Compare Nemotron deployments using a repeatable planning-quality benchmark.
- Add streaming model-plan explanations and execution progress to the UI.
- Expand scope and network policy enforcement across every tool input.
- Improve report export and remediation tracking.
- Add stronger integration tests for the full Nemotron-to-plan-to-report path.