NebiusGuard — Streaming Security for Nemotron

Inspiration

AI agents powered by large language models like NVIDIA Nemotron-4-340B require robust runtime security, strict tool boundary enforcement, and real-time PII/credential sanitization when deployed via streaming endpoints on Nebius AI Studio.

What it does

NebiusGuard is a zero-latency streaming security proxy and firewall designed specifically for NVIDIA Nemotron. It acts as an active middleware guardrail ensuring secure, compliant, and tamper-evident AI interactions.

Key features include:

1. Tool Boundary Validation: Enforces strict JSON schema validation and authorization checks for model tool calls.
2. SSRF Prevention: Inspects and sanitizes all generated URLs and network calls before execution.
3. SSE Token Leak Interception: Real-time inspection of Server-Sent Events (SSE) token streams to prevent API key and credential leaks.
4. Exact Prompt Caching & Tamper-Evident Audit Hash Chain: Cryptographically logs every interaction into an immutable audit ledger.

How we built it

Built with Node.js ESM, leveraging high-performance streaming parsers, cryptographic hash chains, and rigorous unit test suites (8/8 tests passing in 50ms across Node 20, 22, and 24).

Challenges we ran into

Handling high-throughput streaming Server-Sent Events with zero latency overhead while ensuring robust regex and AST-based AST inspection for credential leakage.

Accomplishments that we're proud of

• 100% test pass rate across all security vectors.
• Seamless integration with Nebius AI Studio & NVIDIA Nemotron.
• Production-ready GitHub repository: sidsri14/nebiusguard.

What we learned

Advanced runtime guardrails are essential for enterprise-grade autonomous AI deployments.

Built With

• api
• javascript
• node.js
• nvidia-nemotron
• security