CARE-Vision

Adaptive agentic vision for safer independent living. OpenCV 5 + COOL on AWS Graviton drives targeted sensing to investigate possible incidents and reduce unnecessary video transmission.

Comment

Inspiration

Independent living creates a difficult safety trade-off: a possible incident may need more visual evidence, while continuous high-quality monitoring consumes bandwidth, cloud compute, and privacy. CARE-Vision explores whether a vision system can decide what it should observe next before escalating a concern.

The intended beneficiaries are older adults living independently and the people who support them. The current application is an experimental safety-support prototype. Its evaluation uses development recordings; incident accuracy in the intended population remains unmeasured.

What it does

CARE-Vision maintains a perception–decision–action loop. A lightweight Edge Gateway captures video, maintains a dense rolling buffer, samples frames, and executes sensing commands. The main computer vision workload and a deterministic Agentic Controller run on AWS Graviton using OpenCV 5 + COOL.

Visual observations can cause the controller to request a different transmission rate, a person region of interest, a full frame, or recent buffered evidence. New evidence is processed again before the controller continues observing, records recovery, or emits an explicitly enabled test alert. Camera capture rate and effective transmission rate are tracked separately.

Camera/video → Edge sampling and buffer → AWS Graviton / OpenCV 5 + COOL → structured observation → agent decision → sensing command → new visual evidence

How I built it

I built CARE-Vision as an individual project in Argentina, supported by an AWS Compute Grant for the OpenCV AI Competition 2026.

The implementation uses Python, persistent WebSocket communication, bounded queues, capture timestamps, and explicit command acknowledgements. The cloud pipeline combines person detection and tracking, posture and temporal evidence, and an experimental pose-assisted ROI workflow. The evaluated cloud configuration uses an m8g.4xlarge Graviton instance, an OpenCV 5 development build supplied with the COOL SDK, YOLOX-S with the CLASSIC engine, and eight OpenCV threads.

The controller uses reproducible rules and temporal state. Each sensing action is linked to the observation that caused it, its acknowledgement, and subsequent visual observations. The gateway performs acquisition, representation, encoding, and transport; inference remains in AWS.

Results so far

The latest paired sensing study, completed on 25 October, ran 18 Graviton/OpenCV 5 + COOL replays: three repeated fixed5 versus adaptive2/5 pairs on each of three known recordings (recovery, no recovery and sitting). Both arms used the same person-only detector/tracker and incident rules; pose, ROI and seated closure were disabled in both arms.

  • 8 of 9 pairs met the preregistered outcome/alert and additional-delay gates.
  • Recovery: median JPEG payload reduction 53.54%; direct cloud CPU reduction 53.12%.
  • No recovery: median JPEG payload reduction 53.47%; direct cloud CPU reduction 53.07%.
  • Sitting: median JPEG payload reduction 47.17%; direct cloud CPU reduction 46.61%.

The failed adaptive no-recovery replay reconnected, missed the expected test alert and ended VERIFYING. Of 171 locally completed sends, 160 were received and processed. That failure remains included; complete outcome equivalence is not established. Sitting remained UNRESOLVED without alerts in both arms, a retained limitation rather than correct benign classification. JPEG payload excludes transport overhead; CPU comes from process counters, not AWS billing. Known recordings and repeated replays do not establish general fall accuracy or a sustained 30 FPS/1080p comparison.

A separate 17 October standard OpenCV 5 versus COOL comparison did not demonstrate a COOL speedup. Original full-pipeline output equivalence failed in 102/240 frames. An isolated shared-input-blob control retained 1,920 exact tensor/detection comparisons, with forward medians of 67.701 ms standard and 67.733 ms COOL. This control does not establish full-pipeline equivalence.

An earlier constrained-link regression evaluated five conditions across four existing recordings: recovery, continued observation without recovery, a separate test-alert condition and two posture/ROI scenarios. All five passed its original regression auditor: 414 frames sent/received/processed, no declared drops or connection errors, and 20 checked commands with causality, acknowledgement and subsequent-perception checks. The alert was test-only. This earlier profile used 1280×720 WebP quality 80, a regular full-frame cap of 1.34 FPS, 112 KiB/s application admission and a 128 KiB/s local TCP proxy before the SSH tunnel. It is separate from the fixed5/adaptive2/5 paired study; requested and achieved rates remain distinct.

The public evidence viewer presents saved recovery, no-recovery and sitting episodes, with recorded decisions and controller replay labeled separately. The 3:53 demo video presents earlier development work; the attached technical report and award-evidence packet contain the later paired study, methods, source/input hashes and retained failures. Independent incident precision, recall, clinical benefit and general network resilience remain unmeasured.

Challenges and lessons learned

A stalled pong exposed how transport backlog can interrupt an otherwise responsive vision pipeline. Reducing bytes alone was insufficient: a JPEG quality-25 profile missed the expected test alert, and a WebP quality-50 profile lost the original track and could not establish recovery. Both profiles passed transport checks but failed one functional condition.

In that earlier constrained-link experiment, the higher-quality, regularly sampled profile passed all five conditions. Quality and sampling changed together, so these experiments do not isolate the contribution of either factor. Earlier failures remain part of the evidence.

The later paired study exposed a reconnect-related missed alert. A separate controlled application-close probe evaluated a candidate that persists and exposes a human-review-required status before fresh evidence. It preserves unlinked identity and does not restore incident sensitivity or fabricate an alert. This candidate is separate from the production core, and the original 8/9 study result remains unchanged.

COOL execution on Graviton is documented. The baseline comparisons have not demonstrated a COOL speedup. The native SDK was already installed on the evaluation host, and provisioning it from a blank environment remains an open reproducibility limitation.

Responsible use and next steps

CARE-Vision is a research prototype for safety support. Medical diagnosis, validated fall detection, and guaranteed caregiver assistance are outside its demonstrated scope. Alerts remain test-only, with no external caregiver notification. Missing evidence, connectivity loss, or a new person identity must not be treated as proof of recovery.

Presentation copies can apply head blur after inference; this does not anonymize the original inputs or automatically protect a live screen-share. Personal recordings remain restricted to authorized private judge access.

Next steps are to freeze the chosen delivery profile, evaluate newly recorded everyday activities with annotations defined before examining predictions, strengthen network and uncertainty evaluation, and rehearse the final judge demonstration. The implementation, measured limitations, and failed experiments are documented alongside pinned dependencies and reproducible run instructions.

Share this project:

Updates

Submission history