third-guard

Logo

Inspiration

The idea started with a lived experience. One of our teammates spent summers working as a lifeguard, and knew firsthand the impossible math of the job: a single pair of eyes scanning a crowded pool, where a drowning can happen silently in under 30 seconds, often within arm's reach of other swimmers. Real drowning rarely looks like the dramatic splashing people expect — it's the quiet, vertical "Instinctive Drowning Response," which is exactly what's easy to miss.

When our group started exploring what QNX — a real-time operating system built for safety-critical systems like cars and medical devices — could do beyond its usual domains, that lifeguard experience clicked into place. Human attention has hard biological limits: fatigue, blind spots, distraction, and the sheer impossibility of watching fifty people at once. An AI system has none of those. It can track every swimmer in the pool simultaneously, every frame, without ever looking away. We were inspired to build Third Guard because we saw a way to point serious, dependable technology at a genuinely present, life-or-death problem — not to replace the lifeguard, but to give them a tireless second (and third) set of eyes.

What it does

Third Guard is an AI-powered water-safety system that turns a live pool camera feed into a real-time distress-monitoring dashboard for lifeguards.

Concretely, it:

Watches the whole pool at once — continuously detecting and tracking every swimmer in frame, each with a persistent identity, so no one drops off the radar. Reads body language, not just position — running pose estimation on each swimmer to understand posture and movement, not merely where they are. Recognizes the signature of distress — scoring interpretable danger signals drawn from real drowning behavior: a vertical torso, the head sinking low toward the waterline, an absence of forward progress, and vertical bobbing. Waits before it cries wolf — only escalating to an alert once the distress pattern persists for a swimmer over time, so a single odd frame doesn't trigger a false alarm. Makes the emergency impossible to miss — surfacing a live annotated dashboard with green tracking boxes, skeleton overlays, and held red POTENTIAL DANGER / DROWNING ALERT markers, a persistent status banner, and a swimmer count, all viewable in a browser and recorded to video for review. The result is a dependable real-time tool that lets a lifeguard instantly pinpoint who is in trouble and where — augmenting their judgment rather than replacing it.

How we built it

Building Third Guard meant engineering a complete pipeline from the camera lens all the way to the lifeguard's screen — and it forced us to make deliberate decisions at every layer.

Capture (QNX + Raspberry Pi): A Raspberry Pi 5 with Camera Module 3 runs the capture side on QNX, using the QNX Sensor Framework to pull frames at roughly 28 FPS at 2304×1296. The Pi's job is intentionally narrow — grab frames and stream them out as MJPEG — which kept the safety-critical edge device lean. Transport (Pi → workstation): Rather than fight to build TensorFlow for QNX, we split the system: the Pi streams video over an SSH tunnel to a workstation that hosts the heavy AI. This turned a limitation into a clean architectural boundary. Processing (OpenCV + TensorFlow Lite): On the workstation, OpenCV decodes the stream and a threaded pipeline runs the intelligence — a TFLite SSD-MobileNet detector finds swimmers, Google's MoveNet estimates each one's pose, a lightweight IOU-based (SORT-lite) tracker keeps identities stable across frames, and a custom distress analyzer fuses posture and motion into a debounced risk score. Everything runs on int8-quantized models on CPU via the XNNPACK delegate, so no GPU is required. Concurrency (so it never stalls): Capture and inference run on separate threads joined by a fixed-size, drop-oldest ring buffer. If inference briefly falls behind the camera, we discard stale frames instead of letting latency pile up — the AI always analyzes the freshest view of the pool. Interface (the dashboard): Finally, an MJPEG server streams an annotated view to any browser — boxes, MoveNet skeletons, held danger overlays, a status banner, swimmer count, and live AI FPS — with the annotated footage also saved to MP4 for after-the-fact review. In short, the project was genuinely end-to-end: hardware capture, real-time OS integration, network transport, multi-stage computer vision, and a human-facing alerting UI, all wired into one working system.

Challenges we ran into

By far our steepest challenge was integration — getting our C++ code, QNX, and the Raspberry Pi to cooperate as one system.

Cross-compilation was the real boss fight. Producing binaries that ran correctly on QNX/aarch64 meant wrangling a cross-compilation toolchain and a web of interdependencies across OpenCV, TensorFlow Lite, and the QNX libraries. We ultimately had to pin an exact TensorFlow 2.16.1 source checkout and carefully control what got built where to get a reproducible build at all. The camera didn't work the "obvious" way. We learned the hard way that QNX doesn't expose cameras through V4L2/OpenCV the way Linux does — it uses its own Sensor Framework. The standard capture approach simply doesn't port, and discovering the platform-native path was what finally unblocked live video. Deciding where the AI should live. Building the full TensorFlow stack for QNX proved impractical, which pushed us toward the split architecture — Pi/QNX captures and streams, the workstation runs inference. What began as a painful constraint became one of our cleaner design decisions. Making real-time actually real-time. Early on, frames piled up whenever inference lagged, and the feed drifted out of sync. Solving that required rethinking the data flow into a threaded producer/consumer with a drop-oldest buffer — proving that for a live safety system, architecture matters as much as model accuracy. Tuning alerts to be trustworthy. Getting the distress detection to fire on genuine danger without spamming false alarms took real iteration on thresholds and on requiring the distress condition to persist before escalating — the difference between a jittery demo and something a lifeguard could rely on.