QNX-Aegis: AI Agent & Real-Time Microkernel Bridge

Real-time AI agent & telemetry bridge for QNX Neutrino 8.0. Inspects microkernel IPC via MCP, detects MsgSend() deadlocks, benchmarks ClockCycles, and autonomously hot-patches via QCC.

Comment

Inspiration

QNX Neutrino RTOS 8.0 is the gold-standard real-time microkernel operating system powering mission-critical infrastructure worldwide—from software-defined vehicles (SDVs) and medical robotics to avionics and industrial automation. Unlike monolithic kernels (Linux, Windows) where drivers and subsystems run in Ring 0, QNX operates on an elegant true-microkernel paradigm where nearly all services execute in user space (Ring 3), communicating strictly via synchronous, unbuffered Inter-Process Communication (IPC) (MsgSend, MsgReceive, MsgReply).

However, diagnosing complex IPC deadlocks, priority inversions, and multi-threaded race conditions in modern embedded systems has historically required deep manual expertise, static serial logs, and disjointed command-line utilities (pidin, tracelogger).

With the emergence of Autonomous AI Agents (Claude Desktop, Cursor, Antigravity) powered by Anthropic's open Model Context Protocol (MCP), we asked a transformative question:
What if autonomous AI coding assistants could directly "see", inspect, diagnose, and hot-patch live QNX 8.0 microkernel states in real time?

That vision inspired QNX-Aegis: an open-source, full-stack AI Agent & Real-Time Microkernel Bridge that turns dense RTOS telemetry into an intuitive, high-velocity developer workstation.

What it does

QNX-Aegis bridges live QNX Neutrino 8.0 microkernel telemetry with modern AI agents and host development environments through four tightly integrated pillars:

  1. Native QNX 8.0 Resource Manager (/dev/qnx_inspector):
    A lightweight, zero-copy C daemon running directly in user space on the QNX target. It attaches to the QNX pathname space via resmgr_attach(), queries thread states from the Process Manager (/proc/<pid>/as) via native devctl() commands (DCMD_PROC_TIDSTATUS, DCMD_PROC_INFO), and streams structured JSON telemetry without intrusive kernel overhead.

  2. Standardized Model Context Protocol (MCP) Server:
    Exposes 6 production-grade JSON-RPC 2.0 tools allowing any MCP-compliant AI assistant to interact with QNX 8.0:

    • qnx_verify_environment: Target connectivity, SSH health, and host QCC cross-compiler status.
    • qnx_compile_source: Autonomous host compilation of C sources via qcc -Vgcc_ntox86_64.
    • qnx_deploy_and_execute: Seamless SCP binary deployment and detached remote execution.
    • qnx_inspect_microkernel: Live inspection of all 150+ threads, channels (chid), connections (coid), and blocking states.
    • qnx_detect_ipc_deadlock: Graph-theoretic deadlock cycle detection and priority inversion auditing.
    • qnx_read_slog2: Real-time extraction of system logs from QNX System Logger 2 (slog2) ring buffers.

  3. ZOC Terminal + btop Workstation Dashboard:
    A high-contrast, cyberpunk terminal UI featuring:

    • ¹kernel-cpu Panel: Live dual-tone Unicode Braille sparklines (⣀⣤⣶⣿) tracking IPC activity over 60-poll windows alongside ASCII block meters (■■■■■■□□).
    • ²ipc-topology Panel: Interactive Cytoscape.js IPC graph rendering live message-passing topologies with state-specific glowing nodes (RUNNING, RECEIVE, SEND, REPLY).
    • ⁴proc (pidin) Panel: Real-time process and thread table with instant search filtering, keyboard navigation, and two-way click synchronization with the Cytoscape graph.
    • ⁵slog2-stream Console: Live ANSI-formatted streaming of native QNX slog2 ring buffers.

  4. Deadlock Detective & Autonomous QCC Hot-Patching:
    Implements Tarjan’s directed cycle detection algorithm on STATE_SEND/STATE_REPLY thread dependency graphs to detect circular wait-for deadlocks and Priority Inversions in $O(V + E)$ time. With a single click (or via autonomous AI agent invocation), QNX-Aegis slays the deadlocked process, recompiles fixed C code on the host via qcc, deploys the ELF binary via SCP, and verifies healthy recovery in under 2 seconds.

  5. Hardware ClockCycles() IPC Benchmark:
    Executes 1,000 synchronous 64-byte MsgSend() round-trips measured directly against hardware CPU cycle counters (ClockCycles()), reporting sub-microsecond average latency and message throughput.

How we built it

We engineered QNX-Aegis from the metal up, ensuring zero simulated data and 100% adherence to QNX Neutrino RTOS 8.0 architectures:

  • Target Microkernel Layer (qnx-target/):

    • Cross-compiled with QNX Software Development Platform (SDP 8.0) using qcc -Vgcc_ntox86_64 -O2 -Wall -Wextra -D_QNX_SOURCE.
    • Built /dev/qnx_inspector using the QNX Resource Manager framework (dispatch_create(), resmgr_attach(), iofunc_func_init(), resmgr_block()).
    • Implemented /proc thread inspection via open("/proc/<pid>/as", O_RDONLY) and devctl(fd, DCMD_PROC_TIDSTATUS, &status, ...).
    • Built synthetic fault injection demos: circular MsgSend() deadlocks (ipc_deadlock_demo.c), healthy non-blocking IPC loops (ipc_healthy_demo.c), structured fault logging (slog2_crash_demo.c), and hardware benchmarking (ipc_benchmark_demo.c).

  • Host Telemetry & AI Bridge Layer (mcp-server/):

    • FastMCP & Python 3.11: Exposes the MCP tools via stdio and SSE transport.
    • Paramiko & SCP Transport: Persistent connection pooling to QEMU port 2222 with automatic target directory provisioning, /data read-write mounting, and root sudo privilege management.
    • FastAPI & WebSockets: Broadcasts real-time microkernel snapshots to connected web clients with non-blocking asyncio.to_thread execution.

  • Developer Cockpit Frontend (dashboard/):

    • Pure Vanilla JavaScript + Tailwind CSS, styled to mirror the authentic aesthetic of ZOC Terminal running btop.
    • Cytoscape.js with force-directed COSE layout, custom neon shaders, and reactive node pulsing for deadlocked entities.
    • Interactive MCP JSON-RPC Studio for live in-browser testing of MCP tools.

Challenges we ran into

  1. QNX /proc Security Permissions: On QNX 8.0, unprivileged users (qnxuser) receive EACCES when attempting to inspect root/system processes via /proc/<pid>/as, which previously blinded the inspector to microkernel threads. We solved this by implementing self-healing root provisioning via sudo within qemu_bridge.py, successfully unlocking visibility into all 150+ live kernel threads.
  2. Persistent Storage vs. Volatile Daemon State: On QNX 8.0, /tmp is symlinked to persistent storage (/data/var/tmp), meaning binaries persist across VM reboots while the RAM-based /dev/qnx_inspector daemon vanishes. We built an automated health probe that detects missing character devices and auto-provisions the root daemon on boot.
  3. Zero-Simulation Guarantee: Embedded engineers must trust their tools. We enforced a strict zero-simulation rule: if the QEMU VM is offline, the dashboard displays a clear disconnected state rather than fabricating synthetic telemetry.
  4. Sub-Microsecond ClockCycles Timing: Converting raw CPU cycle counts into nanoseconds required querying the microkernel system page (SYSPAGE_ENTRY(qtime)->cycles_per_sec), ensuring accurate hardware metrics across varying host CPU virtualization states.

Accomplishments that we're proud of

  • 100% Native QNX 8.0 Technologies: Fully leveraged QNX Resource Managers, /proc devctls, slog2 ring buffers, ClockCycles(), and qcc without relying on generic Linux emulations.
  • First-of-its-Kind MCP Bridge for RTOS: Pioneered the integration of Anthropic's Model Context Protocol with real-time microkernel operating systems, demonstrating autonomous closed-loop AI debugging.
  • Sub-Second AI Remediation: Achieved end-to-end fault detection, AI diagnosis, cross-compilation, deployment, and verification in under 2 seconds.
  • High-Fidelity Developer Experience: Delivered a workstation experience that embedded developers love—combining retro-futuristic TUI aesthetics with interactive graph topology visualizations.

What we learned

  • The unparalleled architectural elegance of the QNX Neutrino message-passing state machine (STATE_SEND, STATE_RECEIVE, STATE_REPLY).
  • The power of QNX Resource Managers in exposing kernel-level diagnostics through standard POSIX file abstractions (cat /dev/qnx_inspector).
  • How modern LLMs, when equipped with deterministic MCP tools and real-time RTOS telemetry, can reason about low-level concurrency bugs that typically take human engineers hours to trace.

What's next for QNX-Aegis: AI Agent & Real-Time Microkernel Bridge

  • Target Architecture Expansion: Cross-compile and validate QNX-Aegis for ARM64 targets (NXP i.MX8, Renesas R-Car, Raspberry Pi 4).
  • Adaptive Partitioning (APS) Visualization: Add telemetry support for QNX Adaptive Partitioning Scheduler, allowing AI agents to detect and resolve CPU starvation across real-time security domains.
  • IDE Native Extension: Package QNX-Aegis as a native VS Code and QNX Momentics IDE sidecar extension.
  • Autonomous Core Dump Analysis: Integrate native QNX dumper post-mortem analysis directly into the MCP tool suite for automated root-cause reporting on crash dumps.

Built With

Share this project:

Updates

Submission history