ClearSky

Clouds shouldn’t end your investigation. ClearSky’s OpenCV 5 agent finds before-and-after satellite images, checks alignment, and helps recover failed comparisons through evidence and human approval.

Comment

Inspiration

What interests me about environmental data is how we get from an observation to a conclusion: what was measured, which assumptions were made, and whether someone else can follow the reasoning for themselves. Satellite imagery makes those questions rather practical. Clouds, shadows, missing observations and small alignment errors can all affect what a comparison appears to show.

With ClearSky, we wanted to make image selection itself inspectable. Before interpreting a change, we wanted to establish which images could support a comparison, and to keep the decisions that led there visible.

What it does

ClearSky is a browser workbench for finding and checking before-and-after Sentinel-2 imagery. A user chooses a location and two date windows, previews available scenes, approves the download and explicitly confirms the analysis request.

The agent inspects imagery, checks whether a pair meets the configured quality and alignment requirements, and uses those measurements to decide what to try next. When recovery requires different dates, it presents the exact proposed change for approval. A grant creates a new request revision; the user still chooses whether to run it. Denial preserves the original request, and neither decision replenishes the mission's remaining allowance.

ClearSky shows why a pair can support a comparison, asks before changing dates, and keeps failed attempts visible. A supported result produces a portable comparison brief containing the selected imagery, registered views, acquisition details, quality measurements, provenance and limitations. The original failure and any granted change remain part of the record.

How we built it

We developed ClearSky as a standalone extension alongside our existing StarSearch codebase, documenting the boundary between pre-existing work and the competition implementation. It uses Python, SQLite and a browser interface built with JavaScript, HTML and CSS. Rasterio reads the prepared imagery; provider scene-classification data and NumPy supply suitability accounting.

OpenCV 5 performs masked ORB feature extraction, descriptor matching, RANSAC alignment, and checks on spatial support, geometry and usable overlap. The accepted transform also produces the registered comparison views.

The executed agent is a bounded deterministic controller. Its next action depends on the measured result. In a real Tokyo case, an OpenCV registration rejection led it to inspect another acquisition and test a different pair. That pair also failed, and the agent returned no supported result. Both attempts remain visible.

We also built a separate AWS worker path around a pinned Docker image, private EC2, S3 and Systems Manager. A successful OpenCV run on AWS has not yet been verified; the current browser workbench runs locally.

Challenges we ran into

One challenge was keeping different kinds of failure distinct. An image rejected for poor coverage, a failed download and an exhausted inspection budget each require a different response. We made those distinctions explicit and kept uncertain work reserved, so an interrupted run could not quietly regain its allowance.

Recovery raised a related question: what evidence can be reused safely? Historical rejection reuse is bound to the same task, inputs, area, configuration and runtime. Accepted images and pair validation remain fresh. We also worked through persistent state, exact date permissions, public imagery preparation and recipient installation.

Accomplishments we're proud of

In one development example in Valencia, both initial requests failed. After the same seven-day extension, the evidence-aware controller completed the revised comparison within the same six-fresh-inspection allowance, while the cold baseline exhausted it. The successful treatment used more total actions and bytes; its contribution was completing that revised request.

We then froze 48 requests across 24 new locations: 16 cities and eight US national-park points. We retained every attempted request, including preparation failures. Thirty-one were analysis-ready, and both treatments supported eight of the 48 attempted requests. Reuse reduced fresh inspections from 106 to 75, but increased total actions and bytes and produced no additional supported completions in this cohort.

We are proud that the product makes these outcomes inspectable, including the cases that did not work. The browser workflow, saved decisions, remaining allowances and exported evidence have also been exercised through restart and recipient-installation checks on the development machine.

What we learned

The broader evaluation made the conditions under which reuse helps much clearer. Avoiding a repeated measurement does not necessarily reduce total work or produce a usable comparison. Likewise, a good fit to the points used for alignment does not establish independent positional accuracy.

Independent accuracy, false-acceptance rates and analyst time savings remain unmeasured. ClearSky currently supports imagery selection and comparison review; environmental diagnosis and prediction require further validation.

What's next

Our next priorities are independent landmark evaluation, testing with analysts on their own requests, and improving preparation where tile boundaries or transport limits prevent a comparison. We also want recovery proposals to reflect which additional imagery is actually available, so a user's permission can lead to a useful next step.

The longer-term aim is to help people examine environmental evidence with a clearer understanding of where it came from, what was checked and what remains uncertain.

Built With

Share this project:

Updates