Bird-safe gap check

Find the gaps in a bird-safe window treatment before the job is signed off.

Comment

Inspiration

Glass collisions kill an estimated 365 to 988 million birds a year in the United States (Loss et al., The Condor 116(1), 2014). Most bird-friendly glass guidance calls for markings on the outside of the glass with no gap wider than 2 inches (50.8 mm). Dot treatments are applied by hand, though. If one dot falls off or never gets placed, the gap on a 40 mm grid of 3/4 inch dots opens to about 61 mm, and on a finished job nobody measures every gap with a ruler. I wanted a way for the installer to check the whole patch with one phone photo before the job is signed off.

What it does

You tape a printed ChArUco calibration sheet on the glass next to the dots, open the live page on a phone, pick the dot size and take a photo. There is nothing to install. The app answers with one of three things:

  • PASS: no gap wider than 2 inches in the area checked.
  • REPAIR: the largest gap in millimetres, and a target on the photo where a dot should go.
  • RETAKE: the reason (dark photo, glare, sheet not found, view too steep) and which part of the glass to aim at.

The next photo goes into the same scan. After a REPAIR it checks the spot again, and when it is fixed the page says "The repaired spot now passes." In the live demo on 24 Oct 2026, a scan reported REPAIR at 62.0 mm with one dot peeled off, then PASS at 40.0 mm after I put it back.

How I built it

The image processing runs on OpenCV 5.0.0.93 in Python 3.12, on AWS Lambda arm64 (Graviton) behind a public function URL.

  1. Rectify to millimetres. cv2.aruco.CharucoDetector finds the board corners, cv2.findHomography with RANSAC fits the glass plane, and cv2.warpPerspective flattens the photo so every pixel has a known size on the glass. The sheet is the only source of scale, so a photo without it gets a RETAKE.
  2. Find the dots. Top-hat and black-hat filters pick out light and dark blobs, and cv2.findContours measures them. A blob counts as a dot only with roundness of 0.80 or more and a diameter 0.7 to 2.0 times the stated dot size. On the development window, real dots measured 0.88 or more and reflection blobs 0.74 or less.
  3. Mark glass it cannot read. Saturated pixels with no texture are glare, and low Laplacian energy is blur or darkness. That glass is marked unknown, and it never counts as covered or as a gap.
  4. Find the largest empty square. cv2.distanceTransform with DIST_C over readable empty glass gives the largest empty square in millimetres, checked against 50.8 mm. Each spot where a 2 inch square fits gets a repair target.
  5. Merge photos on the board plane. Each photo lands in the board's own coordinates, so photos in one scan merge without photo-to-photo registration, using only the pixels each one resolved.
  6. Decide. An ordered rule table (no language model) turns the numbers into the next step and writes a trace that replays to the same decisions. Three RETAKE decisions in a row that do not help end the scan.

The phone page is plain HTML and JavaScript. Photos are processed in memory and never stored. Only the fused masks and the trace go to a private S3 bucket, deleted after 7 days, and CloudWatch gets one JSON line per photo. deploy/deploy.sh builds and deploys the whole stack. The full method is in the technical report.

Challenges I ran into

  • Reflections. Windows reflect sky, trees and whoever is holding the phone. Bright reflection blobs can look like light dots, so the roundness and size limits have to separate them, and saturated glare has to be marked as unreadable.
  • Dense grids. With 3/4 inch dots at 40 mm, the glass between bright dots also shows up as a dark blob. When I combined the two filter outputs, every dot merged into one lattice. Running each polarity on its own fixed it.
  • Unreadable glass. A dark area could be treated as empty or as covered. Marking it unknown and asking for a targeted retake took more work, and it means the app never guesses about glass it cannot read.
  • False PASS results. A code review I did before release found two ways the app could pass when it should not: a grid too sparse to group, and an unreadable repair spot being called fixed. It also found missing input limits. I fixed all of them with tests, and the real photos and phone trace kept their decisions.

Where it still fails

  • In 4 of the 6 readable real photos, one top-row dot was missed against a bright sky reflection. A missed dot reads as empty glass, so the error makes a gap look larger, never smaller, and the false REPAIR shows on the photo.
  • Dark photos get a RETAKE with the reason.
  • The area checked is the dot group plus 3 mm. There is no pane outline step yet.
  • It does not prove the markings are on the exterior surface or that birds see them, and it does not certify a window. The installer confirms placement and signs off.
  • So far I have tested one window and one dot product, with all tuning done in one development session and no held-out set. Traces do not yet record the policy version, and a scan supports one phone at a time.

Accomplishments that I'm proud of

  • On a real window with one dot removed, it measured 60.5 mm against 61 mm expected (2 x 40 mm minus the 19.05 mm dot), and the repair target landed 4.3 mm from the missing dot. With all 16 dots it passed at 43 mm.
  • Rectification fit the 24 ChArUco corners to 0.20 mm RMS on views estimated at 22 to 24 degrees off square.
  • 64 of 64 dots found with 0 false detections on synthetic panes with 6 mm and 19.05 mm dots, and 18 of 18 correct on the gap sweep (30, 40 and 45 mm grids pass, 60, 70 and 90 mm grids are flagged, at 3 resolutions).
  • An earlier phone scan over 5G went REPAIR at 60.5 mm, then RETAKE for a dark photo (68% unreadable), then PASS at 28.5 mm with 16 of 16 dots. The saved trace replays to the same 3 decisions.
  • Warm, a 2400 px photo takes 1.4 to 2.3 s round trip, 0.5 to 1.6 s of it on the server, with 250 MB peak memory (curl runs and CloudWatch).
  • 173 automated tests pass, and CI runs ruff, the tests and an OpenCV 5 version check on every push.

What I learned

A phone photo has no physical scale on its own, and a printed board of known size gives both the scale and the plane of the glass. For this tool, the hardest part was deciding what to do when it cannot read the glass. A plain rule table with a trace made each decision easy to test and explain.

What's next

More windows, more dot products, and a held-out test set. A pane outline step on the phone so the area checked is the whole pane, and the policy version recorded in every trace.

Share this project:

Updates

Submission history