Inspiration

Most wall cracks are harmless, but a crack that keeps widening can mean the building is moving. The standard advice is to measure it every few weeks. Almost nobody does: crack gauges are fiddly, and a phone photo has no scale.

What it does

Measure: The owner sticks a printed 60 mm ArUco marker next to the crack and photographs it weekly in a phone browser. OpenCV 5 measures the crack's maximum and typical width, length and widest point in millimetres, with a stated uncertainty.
Line up: It lines the photo up with last week's, so a moved marker or a different viewpoint is caught.
Decide: An agent on Amazon Bedrock (Nova 2 Lite) calls these OpenCV functions as tools and decides the next step:
- retake with a specific fix (e.g. "hold still and tap to focus")
- close-up of the widest point, which OpenCV located
- save the reading as stable
- ask a structural engineer, who approves through a one-time link before the owner is told to act
Show: Every step, and who decided it (AI model, rules backup or engineer), is visible on the case page.

OpenCV output changes the agent's next tool call. Example: a crack grew from 1.49 to 2.14 mm, but the photo was too coarse to be sure. Instead of raising an alarm, the agent asked for a close-up at the widest point, measured again (2.08 mm, ±0.27), and only then asked an engineer.

How we built it

OpenCV 5:
- ArUco detection gives the scale: $s = \dfrac{60\ \text{mm}}{w_{\text{marker}}\ \text{px}}$, plus the tilt from the homography and a straight-on warp in exact millimetres
- black-hat crack detection with a noise-based threshold, and stain rejection
- thinning plus a Dijkstra main path for length; width as local $\dfrac{\text{area}}{\text{length}}$
- photo quality judged on the marker's edges (blur, tilt, scale)
- alignment: ORB first, and OpenCV 5's new ALIKED + LightGlue networks when ORB finds too few matches (12 of 13 hard real photo pairs lined up, against 9 with ORB alone)
Agent: Bedrock Converse tool use with safety rules enforced in code, a rules backup with the same tools when the model is unavailable, and a full trace of every decision.
AWS: Lambda (Python 3.13) with OpenCV 5 in a layer behind a Function URL; DynamoDB, a private S3 bucket and SNS; least-privilege IAM, a $20 budget alert, and one-command deploy and delete.

Challenges we ran into

OpenCV 5 removed estimatePoseSingleMarkers, so we compute tilt from the homography ourselves.
Thin cracks can't be measured from far away: instead of guessing, the agent asks for a close-up.
Dark stains have rounded edges that look like cracks: we blank out wide dark areas first.
ALIKED + LightGlue at first "lined up" photos of a different wall, because it matched the marker's surroundings. Greying out everything but the wall fixed it.
ALIKED's fixed 640 × 640 input made it less precise than ORB alone, so we measured both on real photos and made it ORB's backup.
The two models (52 MB) don't fit in the Lambda layer next to OpenCV, so they live in S3 and are copied to /tmp at start-up.
Bedrock quota: the agent runs on Amazon Nova 2 Lite. While we recorded the demo, our Bedrock quota was still being approved, so the built-in rules backup made each decision, which also shows the failure handling working.

Accomplishments that we're proud of

On real iPhone photos of our printed test sheet, 13 lines from 0.3 to 3 mm read with a mean error of 0.05 mm (worst 0.11 mm), each inside its stated uncertainty. On synthetic photos, readings repeat within 0.02 mm.
The same photo pipeline leads to six different agent paths, depending on what OpenCV sees.

What we learned

A number with a known right answer is worth more than a confident label: test photos with known widths let us check every change.
"I can't tell" is a result. Much of the agent's value is refusing to measure a bad photo and asking for exactly the photo it needs.
Safety rules belong in code, not in the prompt: the model picks the next tool, but code stops it from closing a case that needs an engineer.
Plan for the model being unavailable: a backup with the same tools and trace kept the app working from day one.