MasqAI

a screenshot of the gateway's console
the Double-Binary defense model
clipped report of the original incident which inspired the MasqAI

Inspiration

MasqAI started with a real security incident: one of our LLM API keys was exposed. Instead of simply revoking it, we monitored the incoming traffic passing through it. What we observed was startling. Autonomous coding agents were routinely leaking environment variables, hardware fingerprints, internal database paths, and personal messaging handles to an untrusted endpoint.

That led to our core realization: an API key is not just a billing token; it is a two-way mirror. Whoever operates an LLM endpoint, relay, or aggregator can profile the user and machine behind the agent.

Most developer security stops at guarding the API key. We asked: what if both the key and the context need protection? We built MasqAI to solve both halves of this equation: shield the context from untrusted models, and prevent API keys from being permanent, static targets.

What it does

MasqAI is a local, fail-closed security gateway for AI agents (127.0.0.1:8787). It sits transparently between an agent (speaking OpenAI or Anthropic Messages protocols) and upstream providers. It secures two assets:

The Context (The Sanitizer):
- Hybrid Detection: Uses fast deterministic regex/entropy filters for secrets and a local LLM (qwen2.5:7b on Ollama) to identify and collapse noisy system intel (like 40-line Get-ChildItem Env: dumps).
- Reversible In-Memory Masking: Sensitive credentials and file paths are swapped with typed placeholders (<KEY_id>, <PATH_id>). Canonical values live only in volatile RAM.
- Bidirectional Tool Restoration: When the model outputs a tool call (e.g., querying a local database), MasqAI inspects the call for malicious exfiltration, then restores real values locally so the agent executes seamlessly.
- Dual-Reality Inspector GUI: A localhost web dashboard displaying raw vs. sanitized prompts side by side, active RAM mappings, and security trace logs in real time.
- Fail-Closed Guarantee: If the local security model errors, times out, or detects prompt manipulation, requests return an HTTP 502 rather than leaking raw data.
The Key (Key Enigma):
- Continuously rotates upstream inference keys via provider management APIs.
- Active keys are leased per request, drained through a grace period so in-flight streaming connections never drop, and promptly revoked.
- Hardware Key Custody: The master management key never has to touch the host computer—it can reside entirely inside an external ESP32 microcontroller enclave.

How we built it

Gateway Core: Built in Python using FastAPI, implementing strict per-API extraction, streaming SSE reconstruction with lookahead buffering for split placeholders, and conversation session tracking with rollback capabilities.
Key Enigma & Custody: Built rotation leasing, provider rate-limiting, and two custody options: an encrypted terminal-unlocked vault on the host (masqai.vault), and an ESP32 hardware module (firmware/esp32/) running MicroPython. The ESP32 enforces lifetime caps and mints short-lived keys over a frame-authenticated, encrypted link.
Automated Daytona Evaluation: To test MasqAI without risking developer machines, we built an evaluation harness across six realistic coding tasks using Daytona sandboxes. Daytona isolates execution with network_block_all=True and provisions per-run synthetic canary tokens to scientifically measure privacy gains and task success rates.

Challenges we ran into

Non-Destructive Context Masking: Agents need real paths to run commands, but the remote model only needs architectural understanding. Restoring placeholders across streaming SSE chunks without blocking responses required building a custom prefix-buffer transformer.
ESP32 Memory Starvation: The original ESP32 board lacked external PSRAM, leaving only ~147 KB of usable heap. Running mbedTLS for TLS 1.2 handshakes with OpenRouter while parsing JSON routinely triggered out-of-memory errors. We solved this by slimming the MicroPython firmware, isolating enrollment into a temporary memory window, and pre-allocating heap space (heap.pregrow()).
Hardware Emergencies at the Hackathon: Mid-event, our ESP32 USB data cable failed. We scrambled to build a lightweight over-the-air (OTA) driver over a phone hotspot to manage deployments wirelessly. But to bootstrap the OTA driver in the first place, we needed a working connection—luckily, a neighboring team building sensor gloves kindly lent us a working board/cable to get our hardware enclave back up and running.
Model Sizing vs. Inline Latency: We initially experimented with larger local models like nemotron:30b, but high spatial complexity and memory constraints led to frequent request timeouts. We tuned and settled on qwen2.5:7b, which provided sub-second inline evaluation while maintaining strict policy adherence.

Accomplishments that we're proud of

A True Zero-Exposure Tool Loop: An agent can be given a database repair task where the remote model reasons over <PATH_1>, while the local tool runs against C:\real\path.db—with zero leaks observed at the upstream relay.
Working Hardware Key Enclave: Successfully offloading master key custody and automated rotation to an external ESP32 microcontroller, verified across a 100+ handshake continuous soak test.
Daytona Security Isolation: "MasqAI protects the context; Daytona contains the actions." Proving our privacy guarantees in fresh, network-blocked cloud sandboxes with dynamic canary tracking.
The Dual-Reality Inspector: Making invisible proxy sanitization and LLM decisions fully visible through a clean, intuitive interface.

What we learned

Intel vs. Secrets: Traditional data loss prevention only looks for API keys. But hostnames, CPU stepping, directory hierarchies, and commit authors combine to create a unique fingerprint that identifies developers just as reliably.
The Security-Utility Tradeoff: Privacy tools cannot simply redact everything. A security layer must balance safety with utility by asking: "Does the remote model actually need this detail to solve the sub-task?"
Spatial & Edge Constraints: Moving cryptographic operations and key custody onto microcontrollers gave us a deep appreciation for low-level memory allocation, TLS overhead, and fail-closed embedded design. This is Andy and I'm suprised

What's next for MasqAI

Scaling Evaluations: Expanding our Daytona canary test suite across larger open-source agent benchmarks (e.g., SWE-bench).
Hardening Against Prompt Injection: Enhancing defenses against adversarial tool outputs that attempt to manipulate the local security LLM into unmasking secrets.
Multi-Provider Key Enigma: Broadening automated key rotation from OpenRouter to native OpenAI, Anthropic, and self-hosted API relays.
Browser Extension: Bringing MasqAI's context masking directly to browser-based AI chats.