GuardianCV: Privacy-First Gait & Fall-Risk Vision

Privacy-first gait and fall-risk screening from ordinary camera video. OpenCV 5 pixel math, Kalman tracking and an auditable perception agent, with honest measured limits. FastAPI, Docker, AWS tested.

Comment

Inspiration

Falls are a major risk in hospital wards and home care, and watching how someone walks is one of the best early signals. But two things get in the way. Streaming raw patient video to the cloud is a privacy liability, and vision pipelines tend to fail quietly in dim corridors, with no warning that the tracker has lost the patient.

What it does

GuardianCV screens gait and fall risk from an ordinary camera feed.

  • Zero-PII shield: the viewport shows only edges and a silhouette on black, with the head redacted. Raw frames are never shown.
  • Pixel-derived metrics: knee flexion, left/right symmetry, sway and tracking confidence come from contours, optical flow, Shannon entropy, Laplacian variance, Canny edge density and hull solidity. No sine waves, no hardcoded confidence.
  • Honest tiers: GREEN / AMBER / RED, and INDETERMINATE when the legs cannot be told apart, instead of a guessed GREEN.
  • Human in the loop: every clinician approve/dismiss goes into an audit ledger, and the FHIR-style prescription requires a clinician's co-signature.

How we built it

OpenCV 5 (MOG2 + Canny segmentation, Farneback optical flow, CLAHE, LUT), a two-leg tracking model, a six-joint Kalman filter bank, and a perception agent that senses frame quality, scores three restoration policies and logs every decision. A FastAPI backend streams frames over WebSocket to a single-page dashboard. It ships as a Docker image, with CI on x86_64 and aarch64 and 21 tests.

On AWS I ran the same pipeline on an Intel c7i-flex.large and a Graviton2 t4g.small (single stream, 100 iterations, 2 vCPU). Intel was about 1.8x faster per frame (32.7 ms vs 58.3 ms). Graviton2 was about 7.6x cheaper per hour and roughly 4x cheaper per processed frame, based on the two reports.

Challenges we ran into

  • Same file, different pixels. Validation passed on x86_64 and failed on arm64. Decoded-frame hashes showed the lossy MP4 decoder produced different pixels per architecture. Committing the test clips as lossless FFV1 made results identical on both.
  • Kalman spikes. The first frames, before the background model learns, produced errors of about 270 px. Gating on valid leg fits removed them, and a process-noise sweep cut median innovation from about 14 px to about 6 px.
  • The agent did not help tracking. I measured it across six clips and four brightness levels. Confidence rose from about 56 to 89 at night, but leg-fit validity did not improve in any of four designs I tried. The confidence score rewards brightness, not usefulness. So tracking now runs on the original frame and restoration is display-only. The agent's value is quality sensing, an auditable log and human control, not better tracking.

Accomplishments that we're proud of

  • Everything is measured from pixels. Knee flexion, symmetry, sway and confidence are computed from contours, optical flow, entropy and edge statistics. No synthetic motion, no hardcoded confidence.
  • An honest tier system. When the two legs can't be told apart, the result is INDETERMINATE, never a guessed GREEN.
  • Found and fixed a cross-architecture bug. Validation passed on x86_64 and failed on arm64. Decoded-frame hashes traced it to the lossy MP4 decoder, and lossless FFV1 clips made both architectures produce identical results.
  • Measured the agent against a baseline, and reported the negative result. Across 6 clips and 4 brightness levels, four restoration designs raised confidence from about 56 to 89 at night but never improved tracking. I changed the design instead of claiming a benefit.
  • Real AWS numbers. The same pipeline on Intel c7i-flex.large and Graviton2 t4g.small: Intel about 1.8x faster per frame, Graviton2 roughly 4x cheaper per processed frame (single stream, one run each).
  • Full engineering loop. FastAPI + WebSocket dashboard, Docker image, 21 tests, and CI on both x86_64 and aarch64.

What we learned

  • A score that looks like a quality measure can reward the wrong thing. My confidence metric weights brightness, so it climbed while tracking got worse. Evaluating against a no-restoration baseline was the only way to see it.
  • "Same file" does not mean "same pixels" across machines. Lossy codecs can decode differently per architecture, which changes fine measurements such as knee-angle peaks. Lossless test data fixed it.
  • Filtering needs trustworthy input. Feeding a Kalman filter every frame, including warm-up frames, produced 270 px error spikes. Gating on valid fits removed them.
  • Negative results are worth reporting. Knowing the agent's restoration doesn't help tracking is more useful than an unsupported claim.
  • Cloud benchmarking has traps: burstable instance types, free-plan instance limits, and the difference between measured and extrapolated capacity.

What's next for GuardianCV: Privacy-First Gait & Fall-Risk Vision

  • Real footage, with consent. Validate on real gait recordings and compare against clinician assessment. Until then the tiers are decision support only, not clinically validated.
  • A restoration that helps tracking. Try approaches that preserve the background model (for example denoising before segmentation, or a learned low-light enhancer) and score them on leg-fit validity, not brightness.
  • Better leg separation. Replace band slicing with a pose model where possible, keeping the Zero-PII rendering on device.
  • A working MCP server that executes the published tool schemas, so an external agent can call the pipeline.
  • Measured scaling on AWS. Run --parallel on Graviton4 (c8g/m8g) and a matching x86 type to get real streams-per-node and cost-per-stream figures.

Built With

Share this project:

Updates

Submission history