Blogger Bear - Visual Adapter

BloggerBear is a serverless AWS agent that researches topics and writes reviewed articles. We're giving it eyes: OpenCV 5 analyzes satellite imagery so changes can become stories. Topic Train Networks

Comment

Team Name: Blogger Bear First Git commit: roughly September 22nd.

Team Bio Hi, I'm Allain, a Sydney-based technology professional interested in platform engineering and technical account management. I develop games in my free time, having launched one on Steam, and apparently I vibe code in hackathons now too. My teammates on this occasion are just AI: Claude, a little bit of Copilot and Amazon Bedrock, and I think Google's AI search deserves a shout-out too.

Inspiration

Blogger Bear looks at trains. Blogger Bear started as an autonomous blogging pipeline for text feeds: it researches topics, notices what changed, drafts articles and holds them for review. We wanted to give it eyes. Where a city builds and where its trains reach drift apart slowly and quietly, and the evidence is sitting in open data nobody reads every week: Sentinel-2 passes over every city every few days, and OpenStreetMap carries every line and station. So we built a visual adapter that watches a city's rail access from space, notices when something moves, checks whether the change is real, and writes about it, with a person approving every word.

I've wondered about Sydney's transport network efficiency and if anything can be learnt from overseas such as leveraging more circular routes or radial designs (like Singapore's network) giving commuters or travelers more options especially on off-shoot or far reaching stations.

What it does (Vision Pipeline):

The Rail Access Monitor watches Sydney, Melbourne, Singapore and Boston, one city a week.

  • Sees: for every new clear Sentinel-2 scene, an OpenCV 5 worker maps built-up density, measures how far every built-up pixel is from a station, outlines the "rail deserts" beyond walking reach, turns the rail network into a graph, and draws the map the article carries.
  • Notices: each measurement is compared with the city's own baseline. If nothing moved, it stops there, with no AI called.
  • Checks: if a number moved, a bounded triage agent on Amazon Bedrock decides whether the change is real or an artefact (cloud, haze, a bad scene). It can ask OpenCV to re-measure, look at earlier scenes and the city's history, and read recent news. Anything unsure counts as an artefact.
  • Writes, then waits for a person: each week it writes a reading of the city, then an article a day from different angles (access, the network's weak points, simulated new links). Every vision article is held until a human approves it.
  • Keeps a record: every measurement, comparison, agent decision and human decision is logged, and the operator can ask about it through an assistant (MCP) or Alexa+.

The same pipeline also counts ships: a new topic watches large vessels in Botany Bay and found 2 on its first pass, on 20 October.

How we built it

  • Claude Code, GitHub Actions, Terraform and an AWS serverless stack.
  • OpenCV 5 worker (Python, opencv-python-headless 5.0): an arm64 AWS Lambda in us-west-2, beside the public Sentinel-2 bucket. It range-reads only the tiles over a city (no GDAL), builds NDBI/NDVI masks with adaptive thresholding and morphology, measures station reach with a labelled distance transform, labels deserts with connected components, and turns OpenStreetMap rail lines into a graph with hit-or-miss thinning and filter2D.
  • Pipeline on AWS: Lambda, Step Functions, EventBridge Scheduler, DynamoDB, S3, CloudFront and CloudWatch, all in Terraform and deployed by GitHub Actions with OIDC.
  • Agent: Claude Haiku 4.5 on Amazon Bedrock, with four tools (look_again, which re-measures with OpenCV, plus previous_scene, site_history and web_context), a tool budget, a hard deadline and a strict JSON verdict. Every error path fails closed.
  • Human control: vision topics are forced to manual review in code, and that can't be switched off.
  • Evaluation harness: pinned scenes (checked by ETag), recorded replays a judge can re-run without AWS, counterfactuals that change only OpenCV's output, and a failure catalogue.
  • COOL: we ran the same worker on a Graviton4 instance with COOL 3.1 from AWS Marketplace, against stock OpenCV 5.0 and 4.14 on the same machine, with a build fingerprint guard that refuses to run on the wrong build.

Challenges we ran into

  • Clouds pretend to be cities. Cloud and haze inflate the built-up mask, so we added scene-level coverage floors and made the agent's default answer "artefact".
  • Lambda limits: a whole city in 1 vCPU, about 1 GB of memory and a time budget meant range-reading only the tiles we need and capping the analysis window.
  • Overpass (OpenStreetMap) is rate-limited and slow at times. Networks are cached for 30 days and failed refreshes are retried.
  • Making a stochastic agent auditable: we built recorded replays and counterfactuals so a decision can be re-checked on any machine.
  • COOL crashed with more than one thread. On our real scenes COOL aborted with heap corruption at 2+ threads. We bisected all 16 OpenCV functions the rail path uses down to connectedComponentsWithStats, found a one-thread workaround that still gives identical results, and drafted a report for the vendor.

Accomplishments that we're proud of

  • A working perception, decision, action and human loop on real satellite data, running in two environments on AWS.
  • Reproducible agent traces: three real replays with the deployed worker and Bedrock re-run exactly without AWS. Every decision matches, and every OpenCV measurement recomputed on an x86 laptop agrees with the Graviton worker's within tolerance.
  • Fails safely: all 8 real failure cases (cloud, missing data, smoke, a site off the scene, an oversized window, an unreadable file, a URL off the allow-list, the wrong OpenCV build) behaved exactly as designed.
  • An honest COOL benchmark: COOL 3.1 gave identical results to stock OpenCV 5.0 (parity pass, no pixel different), at 1.028× speed with a 95% interval of 0.964–1.037. That's not a measurable speed-up on this workload, and we can say why: OpenCV is about 11% of the worker's CPU time.
  • Every number in the technical report cites a committed run file.
  • A measured agent: on 13 labelled cases × 5 runs with the real model, the triage agent scored a balanced accuracy of 0.95 (caught 90% of real changes and 100% of artefacts) against 0.50 for simple baselines, failed closed in all 15 failure runs, and cost under AUD 1 for the whole evaluation.

What we learned

  • On a real workload, the OpenCV kernels are often not the bottleneck: decompressing imagery, Python and numpy took most of the time. Profile before you optimise.
  • The cheapest agent call is the one you don't make. Diffing against a baseline first means the model only sees real candidates.
  • "Fail closed" has to be designed in from the start: timeouts, malformed answers, missing models and cloudy scenes all need a defined, safe outcome.
  • A vendor-optimised build can be correct and still behave differently. Parity checks and fingerprinting caught what a speed number alone would have hidden.

What's next for Blogger Bear - Visual Adapter

  • More cities, more visual articles and workflows, and possibly AI-generated imagery tools.
  • Prompts as versioned assets, with model evals.
  • Other Transportation factored in i.e. Bus Routes

Built With

Share this project:

Updates

Submission history