STRATA — Version Control for the Physical World

OpenCV 5 change detection between two visits of a place: typed events, calibrated confidence, coverage of what it did not see. Human-gated.

Comment

What it does

Photograph a place. Photograph it again later. STRATA tells you what changed, in the sense an engineer would accept: which object left, which arrived, which moved, which was altered — each with a location, a measurement, a confidence, and the reasoning that produced it.

Two visits of the same place are never the same photograph. The camera moves, the light shifts, the exposure drifts. Subtract them and you get the difference between two cameras, not the difference between two moments. So the problem is solved in the order it has to be solved: register, normalise the photometry, detect what genuinely differs, then resolve regions of difference into typed events.

It then does the part most systems skip. It reports what it did not see. Regions it could not observe well enough to compare are marked unresolved and turned into a specific request for another capture, rather than folded into a confident "nothing changed". Nothing enters a site's history until a person approves it.

The result has the properties engineers already trust in a repository: an immutable baseline, a diff against it, typed changes, provenance for every claim, and an append-only history that includes the scans that found nothing.

OpenCV 5

The vision core is written against OpenCV 5.0.0 (opencv-contrib-python==5.0.0.93) and uses APIs that do not exist in 4.x:

  • cv::ALIKED — the CNN keypoint detector and descriptor introduced in OpenCV 5, run at up to 2000 keypoints with 128-dimensional descriptors.
  • cv::LightGlueMatcher — attention-based matching with a learned confidence per correspondence, given spatial context through setPairInfo() before match().
  • cv2.USAC_MAGSAC — robust estimation under OpenCV 5's USAC framework. Registration lands at 0.665 px mean corner error across the benchmark.
  • The classical path (SIFT + FLANN) is a working fallback, not a stub: with no weights on disk the whole product still runs, and every result reports which matcher actually executed.

Two guards exist because of measured failures rather than intuition. LightGlue divides by zero on a featureless image, and on Windows that is a fatal C-level exception — no Python handler runs, and the service process dies with it. A single frame of a blank wall would take a deployment down. The condition is invisible in the features (ALIKED returns 2000 well-spread keypoints with normal descriptors on a uniform frame), so the guard tests the image, using mean absolute Laplacian as a structure energy, and it sits before the call because try/except cannot survive a failure that never unwinds the stack. Measured separation: uniform grey 0.0000, a single small square 0.43, faint sensor noise 5.03, a real scene 15.6–20.2.

The agent

A five-phase loop — observe → decide → act → gate → finish — where the vision output changes what happens next. From a live run, verbatim from the persisted trace:

[observe] scan            by=pipeline      "4 changes; coverage 30%; registration confidence 0.92"
[decide ] request_view    by=policy-rules  "Only 30% of the frame is well observed, against a 60% bar"
[act    ] request_view    by=loop          "Requested a further capture, but none was available"
[decide ] mark_uncertain  by=policy-rules  "…rather than accept a scan that implies the unseen areas were unchanged"
[decide ] finalize        by=policy-rules  "Committing this as the canonical record — which requires your approval"
[gate   ] approval        by=human

The decision layer is a deterministic policy by default. A model may be consulted at exactly one call site, to justify why another view is wanted; if it fails for any reason the run degrades to the rules with the error recorded in the trace. Nothing in the product requires a model to be reachable — which is why the demo runs with no configuration and no API key at all.

A requested capture changes what the system observes: each further frame of the second visit is matched, registered with the same estimator, warped into the canonical frame, and credited to coverage — and a frame that will not register is reported as attempted rather than counted. Coverage accumulates as a union, not a mean, so a second look can only ever add evidence.

AWS

The vision workload runs on AWS Graviton (arm64): Lambda, S3 for session objects and the ONNX feature weights, DynamoDB for the append-only ledger, CloudWatch for logs and per-stage metrics. The hybrid split is stated rather than hidden — development and the interactive console on x86, the vision workload on the Arm path — and the reason is technical, not decorative: the dominant cost is image processing (resize, warp, Sobel gradients, morphology, contour extraction), which is what OpenCV 5 accelerates on AArch64 through the KleidiCV HAL, and the workload is embarrassingly parallel across frame pairs.

Two deployment routes are committed. deploy/deploy.sh builds an arm64 container with docker buildx; deploy/deploy.py ships the same handler to the same architecture as a zip resolved from PyPI's aarch64 wheels, which is the route this machine could actually run. Both are idempotent and have a --dry-run that prints every command without touching an account.

It has been run. deploy/verify.py --record against a live account on 2026-10-20, receipt committed at deploy/verify-report.json, verdict PASS:
Execution environment arm64 (Graviton), python3.12, 3008 MB
Vision stack on the box neural-lightglue — ALIKED + LightGlue, weights fetched from S3 into /tmp/models
Registration homography, confidence 0.9231, 1174 inliers of 1210 matches
Change set 4 typed events (added, missing, state_changed)
Latency 8779 ms inside the function, 9156 ms round trip from outside; match 4941 ms is the bulk
Ledger row read back from DynamoDB: lambda/ledger/verify-bay-4 / 20261020-012803-…json
Public API POST /api/sessions then /rescan over HTTPS: 4 events, storage: s3
Live endpoint https://c3rdzrdswf.execute-api.us-east-1.amazonaws.com/api/health

What has not run is the Graviton4 benchmark on EC2, so no Arm-versus-x86 ratio is published. The harness that would produce it is committed and the protocol is documented; publishing a number that was not measured would contradict the point of the project. Two findings that only a real deployment could produce are recorded in docs/DEPLOYMENT.md §2: opencv-contrib-python does not import on Lambda at all (it links libxcb, which the runtime has no copy of — the deployment uses the headless wheel at the same pinned version), and an anonymous Lambda Function URL is refused in this account even with the canonical public resource policy, which is why the endpoint sits behind API Gateway instead.

Evaluation

24 synthetic scenes at 960×720 generated from seeds, each with known mutations and a known camera pose, plus 12 still pairs where nothing changed. Every figure below is regenerated from the committed artefact by scripts/figures.py, and a test fails if a document stops matching the run.

Metric Result
Registration error 0.665 px mean corner error
Registration confidence 0.9008
Precision 0.9375 (macro; micro 0.9318)
Recall 0.8542 (micro 0.8542)
F1 0.8875 (micro 0.8913)
Event-kind accuracy 0.6979
Confident false positives on unchanged pairs 0.0 per pair, 12 pairs

A reported box counts as the same change as an expected one at IoU ≥ 0.25.

The still-pair number is the one that decides whether the system is usable. A detector that finds things is not hard; one that reliably finds nothing when nothing changed is what makes it deployable, and it is the number most systems do not report.

Reproduced independently. Installing requirements.lock in a clean environment and running python scripts/evaluate.py --cases 24 reproduces the committed artefact case for case, aggregate and still-pair block identical (392.2 s against the recorded 387.5 s). The suite — 481 tests, 79 of which run the real pipeline end to end — passes on the locked set, and CI re-runs the benchmark and gates it against floors on every push.

What does not work

  • Event labels are weaker than event locations. Kind accuracy is 0.6979 against F1 0.8875. The two are reported separately rather than blended.
  • Ten of 24 cases merge two nearby changes into one region. The change is not lost; it is not split. Documented and pinned by a test that fails if the behaviour changes.
  • One seed fragments a move destination into two regions, costing precision on that case, because the polarity guard correctly refuses to fuse opposite-polarity evidence but splits a genuine arrival whose fragments disagree locally.
  • The second geometric model is not exercised by the benchmark. prefer_planar ships True, so all 24 cases registered as homographies and no model-comparison note was recorded. The fundamental-matrix branch is real and is covered by tests that turn it on; the benchmark contains no scene with enough depth structure to need it.
  • No Arm-versus-x86 measurement. The workload genuinely runs on Arm today - the receipt above is from a Graviton function - but the COOL benchmark on a Graviton4 EC2 instance has not been run, so no speed-up ratio is published.
  • The evaluation data is synthetic. Real captures can be uploaded and go through the same code path, but they carry no answer key, so no real-world accuracy is claimed.
  • No metric scale. Output is in canonical pixels: it says an object moved and roughly how far, not how many centimetres.

Responsible use

STRATA observes physical spaces, so consent and data handling are part of the design rather than a disclaimer. It has no face model, no person identification, no biometric template and no behavioural inference — these are structural properties, not policy promises: the schema has nowhere to put a person-level signal. Capture guidance is an instruction to a human holding a camera, not autonomous acquisition. Every write is human-gated and attributed (decision_made_by: human), and the history is append-only, so the record of what was observed is itself observable. Uncertainty is a first-class output: "I could not see this well enough" is preferred over a confident guess, because a confident negative over ground nobody photographed is actionable and wrong.

Run it

Or don't install anything: the app is live at https://strata-console-rouge.vercel.app. Pick a sample site, press Run rescan, and the pipeline registers the two visits on AWS Graviton and returns the typed change set, the coverage it could not see, and the approval gate.

git clone https://github.com/JinYi123456/strata.git
cd strata
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python scripts/serve.py            # → http://127.0.0.1:8000

No configuration, no API keys, no model download to start: five sample sites are generated on first run and go through the same pipeline as an uploaded capture.

pip install -r requirements.lock          # the exact measured set (Python 3.12+)
python scripts/fetch_models.py            # ~52 MB, digest-verified
python scripts/evaluate.py --cases 24     # regenerate every published figure
pytest                                    # 402 fast tests
pytest -m slow                            # 79 end-to-end tests
bash scripts/verify_ui.sh                 # drive the live app in a real browser

Built With

Share this project:

Updates

Submission history