HydraBench

An autonomous multi-agent security sandbox that stress-tests applications, pinpoints root-cause crashes, and delivers verified code fixes.

Comment

💡 The Problem & Core InspirationTraditional integration

testing is plagued by high setup friction. Developers must manually parse code to locate endpoints, determine parameter validation expectations, configure isolated target dependencies, and write custom assertion logic. This friction often results in incomplete coverage, leaving applications vulnerable to unhandled runtime errors and boundary failures.HydraBench changes this dynamic. By utilizing a local-first, repository-aware agentic architecture, it completely automates the pipeline from static code analysis to executable validation. It acts as an autonomous reliability helper that handles the heavy lifting of discovery, isolation, and post-mortem breakdown—ensuring errors are identified locally before hitting production environments.🛠️ System Architecture & Code ComponentsHydraBench separates user configuration, system orchestration, and destructive code execution into isolated modules to maintain total stability.

Code snippetflowchart TD

User[Authorized User] --> Web[Next.js 15 Dashboard] Web --> API[FastAPI Orchestration API] API --> Mapper[Repository Mapper] Mapper --> Plan[Gemini/OpenAI Planner or Local Fallback] API --> Runner[Docker Sandbox Controller] Runner --> App[Temporary Node Application Container] Runner --> Probe[In-Container HTTP Probe] Probe --> API API --> UI[Live SSE Pipeline Updates] Plan --> Remediation[Defensive Remediation Analysis] Remediation --> UI Component BreakdownComponent PathTechnical Responsibility frontend/Next.js 15 & React 19 dashboard handling repository uploads, authorization tracking, route visualizations, real-time pipeline status updates, and diff-view remediation evidence.backend/app/main.pyFastAPI application hosting routes for file uploads, structural analysis, session data persistence, and real-time Server-Sent Event (SSE) streaming.backend/app/repository.pyHandles archive extraction, strict extraction bounds check, source code parsing, and Express/FastAPI route map calculation.backend/app/agents.pyLanguage model integration adapters for Google Gemini and OpenAI, managing structured JSON schemas and boundary-enforced safety prompts.backend/app/orchestrator.pyCentral pipeline state machine governing run flows, log parsing, outcome metrics classification, and remediation hands-off.

backend/app/sandbox.pyDocker SDK lifecycles container automation, applying resource restrictions, managing HTTP probes, and verifying cleanup operations.

🔄 End-to-End Operational WorkflowThe system progresses through a bounded, security-conscious lifecycle to move code from raw source to verified architecture:

  1. Upload & Archive SanitizationThe user provides a local repository ZIP archive and marks explicit testing authorization checkboxes. To prevent denial-of-service vectors, the backend runs strict gate validation: rejecting files $>50\text{ MB}$, archives $>5,000$ internal files, extracted volumes $>200\text{ MB}$, symbolic links, or path-traversal entries. Validated workspaces are securely stored beneath .hydrabench/uploads/.
  2. Static Mapping & Route ExtractionThe repository engine filters for supported files (.js, .ts, .jsx, .tsx, .py) while explicitly ignoring build artifacts, node modules, and version control configurations. It parses files to locate route declarations, automatically navigating internal require() chains and app.use() patterns in Express to reconstruct accurate base endpoint maps.
  3. Model-Driven Test PlanningWhen access is authorized, the system feeds the calculated route footprint and an environment-stripped code snippet to the LLM planner. The model compiles up to 60 distinct HTTP test vectors addressing:Valid/Happy path behaviors.Boundary validations and empty fields.Malformed JSON payloads and incorrect Content-Type headers.Configuration error branches and state dependencies.If the external LLM provider is disconnected, the engine falls back to a built-in deterministic local route map generator.
  4. Sandboxed Containerized ExecutionFor target Node applications containing a package.json, package-lock.json, and an npm start script, the framework copies code into a clean testing workspace, intentionally leaving behind high-risk assets like .env files.Docker spins up an ephemeral validation container image. An internal execution runner waits for application availability, fires the planned HTTP probes sequentially, and streams stdout/stderr output lines back to the orchestrator. Regardless of test outcomes, the container is safely terminated and removed in a finally code execution block.

📊 Result Classification & Self-Healing RemediationWhen execution ends, every single test probe is evaluated based on its status code return family: PASSED: The returned response status matches what the planner expected. (Note: An expected 5xx error code does not log a failure if the test specifically targeted that failure state). VULNERABILITY_DETECTED: The route returned an unhandled, unexpected 5xx server crash log.EXPECTATION_FAILED: The server was reachable, but returned a status family outside of the target design plan.UNREACHABLE: The service crashed or failed to respond within time bounds. JSON{ "test_id": "HB-ERR-004", "target_endpoint": "POST /api/v1/checkout", "outcome": "VULNERABILITY_DETECTED", "failure_signature": "500 Internal Server Error", "root_cause_analysis": "The endpoint does not safely validate parameter input structures before passing them to internal array filters, throwing an unhandled exception that breaks the active thread context.", "remediation_proposal": { "status": "REVIEW_ONLY", "affected_files": ["routes/checkout.js"], "patch_diff": "@@ -12,2 +12,6 @@\n+ if (!req.body.items) { return res.status(400).json({ error: 'Missing items' }); }" } }

For every single anomalous execution state, the Defensive Agent is triggered. It reads the target code file alongside the execution log, identifies the breakdown mechanism, and outputs a clear unified-diff code proposal viewable in the UI. Source modification is never automatic; code updates are presented strictly for developer review to maintain zero user-side regression risks.

🔒 Rigid Security & Safety ControlsBecause HydraBench executes unverified repository configurations, it implements a highly restrictive security profile. Outbound Network Disabling: Containers are initialized with --network none bounds to completely block malicious data exfiltration loops or unauthorized external calls.Read-Only Workspace Systems: The container root file system is mounted strictly as read-only, limiting write operations to a constrained, ephemeral /tmp filesystem. Privilege De-escalation: Linux capabilities are entirely dropped, and no-new-privileges flags are explicitly written into the execution configuration. Resource Containment: Strict CPU core visibility limits, memory threshold caps, and process identifier (PID) counts are enforced per container runtime. Secret Scrubbing: All API keys, environment parameters, and hardcoded tokens are removed from code contents prior to parsing or sandbox construction. 💻 Technical Stack MatrixFrontend User Interface: Next.js 15, React 19, TypeScript, CSS modules/stylesheets.Backend Application Layer: Python 3.12, FastAPI, Pydantic data bindings, Uvicorn execution loop. AI Inference Engineering: Google Gemini API / OpenAI API, driven by strict JSON output structure schemas. Isolation Infrastructure: Docker Desktop / Local engine configurations with optimized Node.js HTTP test probes. File Architecture Processing: Python standard system libraries (zipfile, pathlib, shutil, re).Real-time Streaming Engine: Server-Sent Events (SSE) data pipelines.

Built With

Share this project:

Updates