Inspiration
In real-time systems, a successful build does not always mean a reliable system. A small software change can introduce unexpected latency, missed deadlines, or communication failures—even when average performance looks acceptable.
Developers need a repeatable way to answer a critical question: Does my application still meet its timing requirements after a code change?
PulseProof RT was created to make that question measurable, reproducible, and actionable on QNX 8.0.
What it does
PulseProof RT is an open-source real-time performance and regression testing platform for QNX 8.0.
It combines a native QNX C agent with a Python-based development workbench to run controlled experiments, analyze timing behavior, and automatically evaluate acceptance policies.
Its capabilities include:
- Real-time timing analysis: Measure periodic task response times, wake-up latency, and deadline violations.
- Native QNX IPC testing: Exercise inter-process messaging using QNX channels and synchronous message passing.
- Fault injection and recovery: Introduce controlled server delays, detect kernel timeouts, and verify successful recovery.
- CPU-load experiments: Observe scheduling behavior and tail latency under controlled processor pressure.
- Automated regression detection: Compare compatible executions against unchanged performance requirements.
- Developer-friendly evidence: Explore p50/p95/p99 latency, timelines, empirical distributions, and historical results.
- CI integration: Return explicit exit codes for successful tests, policy failures, and acquisition errors.
- Reproducible reporting: Export raw NDJSON, JSON, CSV, and standalone HTML reports with integrity hashes.
Every reported result is backed by recorded samples. The workbench does not substitute simulated performance measurements for real QNX execution.
A Real Application Regression
To demonstrate practical value, we developed an independent native QNX telemetry service and tested two different application builds.
The first build operates normally. The second introduces a controlled 5 ms delay in its request-processing path.
Both builds are tested against the same 2 ms deadline policy, using the same measurement agent and workload.
Results from the QNX target:
| Measurement | Normal build | Delayed build |
|---|---|---|
| P99 response time | 12.052 µs | 7.017526 ms |
| Deadline violations | 0 | 200 |
| Result | PASS | FAIL |
PulseProof detects the regression automatically and returns exit code 1, making the failure suitable for automated build and release checks.
This demonstrates how a developer can detect a real timing regression before accepting a software change.
How we built it
PulseProof RT was developed using QNX SDP 8.0, with native C11 components compiled using the QNX qcc toolchain.
The QNX 8.0 x86-64 target runs inside VMware Workstation on Windows. The development host communicates with the actual QNX guest through authenticated SSH.
The native implementation uses QNX-specific capabilities including:
ChannelCreate,MsgSend,MsgReceive, andMsgReplyTimerTimeoutfor bounded synchronous communicationname_attachandname_openfor the independent application- Absolute monotonic timing and real-time scheduling
- QNX CPU-time clocks for guest processor utilization estimates
The host application uses Python's standard library, while the interface is implemented using HTML, CSS, JavaScript, and SVG.
The result is a lightweight developer tool without a database, external telemetry service, or third-party runtime Python dependencies.
Challenges we faced
The project presented several engineering challenges.
First, we configured and booted an actual QNX 8.0 VMware target and resolved an OpenSSH component mismatch in the generated image without modifying the installed SDK.
We also addressed Windows OpenSSH compatibility, VMware DHCP lease discovery, and newline conversion issues that could otherwise invalidate downloaded measurement hashes.
Accurate interpretation of real-time results was another important challenge. A timed-out operation is not the same as a completed response that missed its deadline. PulseProof tracks these outcomes separately to prevent misleading statistics.
Finally, we observed that virtualized scheduling introduces environmental noise. We preserved an unexpected failing measurement rather than hiding it or relaxing the acceptance criteria.
Accomplishments
- Built and executed a native C agent on a real QNX 8.0 virtual machine.
- Implemented repeatable timer, CPU-load, IPC, timeout, and recovery scenarios.
- Demonstrated four expected kernel timeouts followed by four successful recoveries.
- Detected a real application regression using an unchanged timing policy.
- Completed 57 host regression tests and 17 local QNX integration checks.
- Passed GitHub Actions across Windows and Ubuntu with Python 3.10 and 3.14.
- Published reproducible measurement evidence, source code, installation instructions, and an English technical codelab.
What we learned
Real-time correctness is about more than average execution speed. Tail latency, deadline policies, fault behavior, and the interpretation of incomplete operations all matter.
We also learned the importance of maintaining complete measurement evidence and comparing only compatible experimental conditions.
Virtualization provides an accessible way to develop and validate native QNX workflows, but results from a shared-host virtual machine must not be presented as physical-hardware timing guarantees.
What's next
Future improvements could include adapters for additional QNX applications, kernel-trace attachments for deeper diagnosis, automated repeated-test campaigns, and validation on physical QNX hardware.
Our long-term goal is to make reproducible real-time acceptance testing a natural part of the QNX development workflow.
Open-source repository: https://github.com/HIZMALI/PulseProofRT
Technical tutorial: https://github.com/HIZMALI/PulseProofRT/blob/main/docs/TUTORIAL.md
Built With
- c11
- css
- git
- github-actions
- html
- javascript
- openssh
- python
- qnx-8.0
- qnx-native-ipc
- qnx-sdp-8.0
- real-time-systems
- svg
- vmware-workstation