Project Title: Mirage: Real-Time Optical Gas Leak Detection & Safety Supervisor
Elevator Pitch
Mirage spots invisible gas leaks using low-cost optical schlieren imaging and triggers a fail-safe valve shutdown in ~1 second—powered by a Raspberry Pi 5, QNX RTOS, and lightweight edge ML.
Inspiration
Gas leaks are completely invisible to the naked eye until disaster strikes. Point gas sensors only trigger when gas physically diffuses to them, while industrial optical gas imaging (OGI) cameras cost upwards of $25,000–$80,000.
Because different gases alter the index of refraction of air, light rays bend slightly when passing through a plume. Laboratories study this using Background-Oriented Schlieren (BOS): by placing a high-contrast dot pattern behind a scene, refractive shifts displace the dots by tiny sub-pixel fractions. We set out to prove that an off-the-shelf $25 camera, a Raspberry Pi 5, and an everyday display showing a dot pattern could run real-time BOS imaging, separate true gas leaks from ambient heat and vibrations, and enforce hard real-time safety guarantees to close an isolation valve.
What It Does
- Optical Detection: A high-contrast dot matrix is placed behind the monitored valve or pipe junction. A Raspberry Pi Camera Module 3 continuously captures the area.
- Sub-Pixel Displacement Tracking: The vision pipeline calculates DIS optical flow against a baseline reference frame, uncovering localized sub-pixel shift fields (0.2–0.5 px) that correspond to refractive plume movement.
- Edge ML Classification: A lightweight TensorFlow Lite model continuously ingests the past 0.4 seconds of vector displacement fields (64×64) and classifies the activity into four distinct states:
clear,gas,heat(e.g., hot drinks, soldering irons), ormotion(e.g., hand movements, table bumps). - Deterministic Fail-Safe Supervisor: An independent safety supervisor task evaluates inference verdicts at 10 Hz. Detecting 3 consecutive gas classifications forces the valve closed via an electromechanical relay.
- Fail-Closed Hard Real-Time Guarantee: If the vision process stalls, crashes, or suffers CPU starvation for more than 250 ms, a dedicated RTOS deadline timer trips the valve shut. Reopening requires human clearance and 5 consecutive
clearverdicts. - Visual Telemetry: Status is surfaced simultaneously via an on-device 6-LED indicator panel and a lightweight live telemetry dashboard.
How We Built It
System Architecture & Flow
[Dot Matrix Display] ──▶ [Pi Camera Module 3]
│
▼
[mirage_vision (QNX 8, Priority 20)]
├── DIS Optical Flow (OpenCV C++)
├── Global Median Subtraction (Vibration Cancellation)
└── 64×64 Displacement Field ──▶ [TFLite Model (Quantized)]
│
▼ 100 ms Verdict via QNX IPC
[supervisor (QNX 8, Priority 50, SCHED_FIFO)]
├── 250 ms Watchdog Timer (Deterministic Failsafe)
├── GPIO Relay Driver ──▶ [Solenoid Valve / Status LEDs]
└── Non-blocking UDP Telemetry ──▶ [Python/Web Dashboard]
Tech Stack
- OS & RTOS Kernel: QNX 8 running on Raspberry Pi 5 (Sensor Framework for camera ingestion, IPC channels, and POSIX timers).
- Computer Vision & Inference: C++, OpenCV (dense DIS optical flow), TensorFlow Lite.
- Safety Supervisor: Pure C (~450 lines of deterministic, statically allocated code running under
SCHED_FIFOpriority 50). - Hardware & Actuation: Raspberry Pi 5, Raspberry Pi Camera Module 3, 5V electromechanical relay board, inline solenoid gas valve, 6-LED status panel.
- Telemetry & UI: Python, WebSockets, HTML5/CSS dashboard.
Rigorous Testing & Results
Evaluated on an experimental rig against recorded validation runs not present in the training set:
- Detection Accuracy: Closed the valve on 47 of 50 distinct lighter/gas releases.
- Median Response Time: 1.0 s from gas release to valve actuation.
- False Trip Rejection: 0 false trips on stationary hot mugs, soldering iron convection, table knocks, and background foot traffic. Only 2 false trips recorded during aggressive, full-frame hand waving.
- Pipeline Latency: ~55 ms camera-to-verdict frame processing (99th percentile).
- Chaos / Fault Injection:
- Under heavy low-priority CPU saturation, the vision and supervisor processes maintained steady 10 Hz operation without frame dropping.
- When artificially starving or killing
mirage_vision, the supervisor's 250 ms watchdog expired, dropping the valve relay to fail-closed immediately.
Challenges We Ran Into
- Microscopic Signal Extraction: Plumes from butane canisters produce displacements of merely 0.2 to 0.5 pixels. Routine ambient lighting flux and minor table vibrations produced larger shifts than the gas itself. We solved this with dynamic reference re-zeroing and global median subtraction across non-plume regions.
- Thermal Convection vs. Gas Discrimination: Simple displacement thresholds could not distinguish gas from hot thermal plumes (e.g., coffee cups). Training our temporal TFLite model on structured time-series displacement patterns resolved distinct fluid dynamics profiles between buoyancy-driven heat convection and expanding gas plumes.
- QNX Real-Time Plumbing: Managing large image arrays across process boundaries required tuning data pipes and optimizing UDP payload chunking to respect QNX's default 9 KB datagram ceilings.
Accomplishments We're Proud Of
- Running full-resolution optical flow and convolutional classification on a Raspberry Pi 5 in under 60 ms per frame.
- Writing a deterministic, static-memory safety supervisor in C that guarantees a physical fail-safe shutdown in under 250 ms during process failure.
- Building an end-to-end optical gas sensing demonstration for less than $120 in parts.
What We Learned
- How to leverage QNX's priority-based preemptive scheduling (
SCHED_FIFO) and kernel messaging for critical hardware safety loops. - The nuances of background-oriented schlieren imaging outside a controlled optics lab bench.
- Practical model quantization and pipeline profiling on embedded ARM hardware.
What's Next for Mirage
- Natural Feature Tracking: Replacing synthetic dot backgrounds with natural surface textures (concrete walls, brick, brushed metal) using multi-scale gradient matching.
- Light-Gas Expansion: Calibrating flow sensitivity to track lighter-than-air gases (methane, natural gas, and hydrogen) that present lower refractive deviations.
- Hardware Watchdog Circuit: Integrating an external discrete hardware watchdog timer (e.g., 555-based or dedicated supervisor IC) to cut relay coil power directly if the SBC loses power or kernel hangs.
Built With
- qns
- raspberrypi5
- tensorflow