Local Human Intent Controller (LHIC)

Inspiration

Most LLM-based browser agents are painfully slow, wildly expensive, and incredibly brittle. Running a setup like WebVoyager or ServiceNow's baselines feels like watching paint dry—taking 20 to 40 seconds just to perform a simple form-fill. Worse, you are paying cloud APIs for trivial actions like clicking "Submit." And the moment a developer shifts a minor DOM class name? The entire pipeline shatters. We wanted to build something that runs at the speed of thought, costs absolutely nothing, and respects user privacy. We asked ourselves: Why call a giant frontier model on the cloud just to click a button we already know how to click? This led us to design LHIC (Local Human Intent Controller)—a hybrid browser automation runtime that combines local deterministic speed with LLM intelligence.

What it does

LHIC is a secure, local-first browser agent that translates natural language human intent into deterministic, local computer actions. It features a unique Fast Path / Slow Path dual-route execution architecture: ​The Fast Path (Local): For standard, repeatable web flows (logins, search, simple navigation), LHIC matches the user's intent against pre-defined, local skills. It runs locally using Playwright in under 35ms with zero API costs. ​The Slow Path (LLM, Fallback): Only activates when a high-risk action is flagged or when encountering completely novel, complex flows. It dynamically delegates to local LLMs or cloud APIs to plan the next action. ​Enterprise-Grade Security: It keeps sensitive sessions safe with KMS-based (AWS/GCP/Vault) signature verification for high-risk clicks, AES-256-GCM encrypted local databases for cookies, automatic PII/credential scrubbing from execution traces, and hardened Docker Seccomp profiles to prevent Chrome sandbox escapes. ​MCP Integration: Out of the box, LHIC is fully compatible with the Model Context Protocol (MCP), meaning you can hook it directly into Cursor, Windsurf, Codex, or Claude Desktop to browse local/private networks safely.

How we built it

We built LHIC using a modern, lightweight, and highly performant TypeScript stack: ​The Core Engine: Powered by Node.js and Playwright with custom BrowserPool context management for thread-safe Chromium pooling and automatic state purification. ​Fuzzy Matching & Memory: Implemented Self-Healing Semantic Locators using a local SQLite database. It keeps track of historical element signatures and uses fuzzy matching on DOM structures, making element targeting completely immune to minor website layout changes. ​Security Shell: Integrated cryptography modules supporting Ed25519 signature checks and AES-256-GCM encryption. We encapsulated the runtime in a minimal Docker environment with custom Seccomp profiles to lock down container permissions. ​Observability: Built-in real-time VNC Screencasting via Chrome DevTools Protocol (CDP) frame broadcasts (running at a configurable 10fps), and telemetry exporting via OpenTelemetry (OTLP) to track execution latency directly in APM dashboards.

Challenges we ran into

Event Loop Bottlenecks: Streaming real-time JPEG screencasts over CDP while running DOM fuzzy matching and database queries in TypeScript was initially choking the Node.js event loop, causing spikes in execution latency. We had to optimize buffer allocations, build an efficient frame-dropping mechanism, and offload CPU-intensive parsing steps. ​DOM Noise vs. Matching Speed: Headless browsers generate massive DOM trees. Parsing and matching these trees against historical SQLite records in real-time without introducing heavy lags was a major hurdle. We designed a custom lightweight selector extraction algorithm that only captures relevant interactive elements, skipping 95% of the static DOM noise. ​Sandboxing CDP: Securing a headless browser while maintaining low-latency bidirectional communication via WebSocket CDP is incredibly tricky. Whitelisting only the necessary system calls in the Docker Seccomp profile without breaking the Chrome GPU and sandbox processes took days of trial and error.

Accomplishments that we're proud of

Median Latency: Getting the Fast Path execution speed down to under 800 milliseconds. It runs so fast that the browser actions feel instantaneous. ​+80% Locator Resilience: Our self-healing fuzzy selector engine successfully matched the correct interactive buttons on popular platforms even after we manually scrambled class names and IDs. ​A Solid Zero-Dependency Dev Experience: The entire project runs with clean Node/TS dependencies, bypassing complex C++ compilation requirements (node-gyp) that make local setup a nightmare for developers.

What we learned

Hybrid Runtimes are the Future: You don't need to choose between "fast & rigid" and "slow & smart." Combining a deterministic fast-path engine with a flexible LLM fallback is the golden standard for building practical, production-ready AI agents. ​Local-First is Viable and Desirable: Running things locally is not just about saving money; it’s about privacy. By keeping session states, cookies, and personal data encrypted on the user's machine, we solved the biggest adoption barrier for AI web agents.

What's next for Local Human Intent Controller (LHIC)

​The Rust Rewrite (NAPI-RS): To squeeze even more performance, we plan to rewrite our heaviest components—such as the HTML parser and SQLite fuzzy matching engine—in Rust, compiling them as native Node.js addons via NAPI-RS. This will push the Fast Path latency down to sub-10ms. ​Deep MCP Integration Expansion: Expanding standard Model Context Protocol support to let you control local smart devices, private intranet dashboards, and complex cloud consoles effortlessly from your favorite local IDEs.

Built With

• javascript
• node.js
• osascript
• python
• typescript