• 

Tabclose

Solo-operator incident monitoring that refuses to cry wolf.

The 30-second version

The problem: Running a service solo means the outage doesn't wake you up, a customer email three hours later does. Single-check uptime tools cry wolf on one flaky ping until you learn to mute them, and by the time you've muted them they're worthless. Full observability stacks need more setup time than a one-person team ever finds.

The solution: Tabclose doesn't trust one witness. Two independently-implemented probes, running in two different regions, have to agree an incident is real inside the same detection window before anything happens. Only once both agree does a Gemini agent read the raw evidence and draft a calm, customer-facing status update to a bucket. It never touches production, never restarts anything, never pages a human directly.

Why it's innovative: Most "AI incident response" demos are one flaky healthcheck bolted to an LLM. Tabclose is built around the one failure mode that actually kills trust in monitoring tools: the false alarm. Corroboration isn't a feature bullet here, it's the whole architecture, and it's why this tool survives past week two instead of getting muted like everything else.

Key features:

• Two-observer corroboration, enforced in code. Probe A (in-process, us-central1) and Probe B (a separately deployed Cloud Function in europe-west1, different HTTP client, different code) must both fail in the same window before an incident exists. A single-region blip is rejected on the spot, with zero artifacts, on purpose.
• Write, don't act. The agent drafts and files evidence; it never restarts services, pages humans, or posts publicly. The blast radius of a bug here is a wrong status draft, not a wrong production action.
• Idempotent by design. Firestore claims mean a crashed run resumes on the next tick instead of double-drafting or losing the incident.
• Free when nothing's wrong. A clean tick exits with no state written and no cost beyond the invocation. You pay attention, and money, only when something is actually broken.
• Live, not staged. Real Cloud Run Jobs, real Cloud Scheduler, real cross-region Cloud Function, real Gemini 2.5 Flash calls, running against production infrastructure right now.

Why I built the thing that watches while I sleep

I run one Cloud Run API by myself. No SRE team, no on-call rotation, just me and whatever I'm doing when it breaks. The failure mode I actually fear isn't the outage, it's finding out about the outage from a support email three hours later because I was asleep, or in a meeting, with the tab closed.

Every uptime tool I looked at solves a different problem than the one I have. They page on a single flaky health check, which trains you to ignore them, or they need enough setup (status pages, integrations, a whole dashboard) that a one-person team never finishes configuring them. I didn't want a tool that pages me on a hunch. I wanted something that would only wake me up once it had already convinced itself twice, and that would leave a written record behind even on the nights I never open the alert.

A monitor that refuses to cry wolf

Tabclose is a monitor that refuses to cry wolf. Every two minutes, Cloud Scheduler fires a Cloud Run Job that probes the target service. If the probe is healthy, the job exits clean: no state written, no cost beyond the invocation, because the overwhelming majority of ticks should be boring and free.

If the probe fails, that's not an incident yet, it's a hypothesis. A second, independently implemented probe, different HTTP client, different code, deployed as its own Cloud Function in europe-west1 instead of living in the same process, checks the same service on its own. Only when both observers report failure inside the same detection window does the run get promoted to a real incident. A single flaky region is rejected on the spot with no artifact at all. Missing a noisy blip is an acceptable cost; training myself to ignore Tabclose is not.

Once, and only once, both observers agree, an ADK LlmAgent backed by Gemini 2.5 Flash reads the raw evidence from both probes, classifies what actually happened (5xx, timeout, connection refused, whatever it was), and drafts a calm, customer-facing status update. That draft gets written to a GCS folder alongside a timeline and both probes' raw evidence. The agent never publishes, tweets, or restarts anything. It writes text to a bucket. A human reads it and decides what to do next. When the service recovers, a later tick writes an all-clear file into the same folder.

The decision that matters most in this system, "is this a real incident," is answered by plain Python arithmetic on two structured dicts, not by a model. No LLM call sits anywhere near that gate. Gemini only gets invoked after the deterministic check has already passed, and that ordering is enforced in code, not by convention: draft_incident() takes the validator's verdict object as its first argument and raises if verdict.passed isn't True. There is no code path in job/main.py that reaches the model before the corroboration check has run.

One tick, start to exit code:

Building for the assumption that nobody's watching

The first real decision was Cloud Run Job, not Service. A Service is built around the assumption that the process lives inside an HTTP request/response cycle. Tabclose's whole premise is that nobody's watching, so there's no request to hang the work off of. job/main.py has no HTTP server at all: main() runs one tick to completion and calls sys.exit(). Every piece of state that needs to survive between ticks, which run_ids have fired, what artifacts already exist, lives outside the process, in Firestore and GCS, not in memory.

The second decision was making the run_id deterministic instead of timestamp-based. floor_to_window() buckets wall-clock time into fixed windows and hashes (service, window_start) into the run_id, so two ticks landing in the same outage window collapse onto the same Firestore claim. A retried Scheduler fire, or a job that crashes after claiming but before writing, can't produce two incident folders for the same outage. I proved this rather than assumed it: I killed the job process mid-run, right after the Firestore claim lands and before the GCS write happens, then let the next Scheduler tick fire naturally. Same run_id, same claim, exactly one folder.

For the agent itself, I kept every ADK session as short as physically possible: one probe read in, one classification call, one draft out, then the process exits. That wasn't a style preference, it came from reading actual adk-python issues before committing to a design. #3559 and #3613 (both since closed) documented a 100-event session cap with no pagination. #5714 (also closed) documented _init_session replaying every prior event on load, which could trip a 429 on a session that had accumulated enough history. None of those are live bugs in the current SDK, but they're exactly what happens if you build a long-lived conversational agent for a background job that's supposed to run forever. So the design has no conversation, just a fresh, short session every two minutes, with anything that needs to persist going to Firestore, never to ADK memory. That's also why agent/incident_agent.py's tool takes probe evidence as explicit keyword arguments instead of relying on Python default values, because the ADK function-calling layer doesn't pick those up.

Tabclose is one of three related submissions built off a shared internal "spine" (agentspine/: idempotency claims, artifact backends, the validator interface, the job runner). Each repo vendors its own copy so git clone and pip install -e . produce a self-contained project with no sibling-directory dependency, and a test in the spine's own suite asserts the copies stay byte-identical across all three repos.

The bug that would have undone the whole point

The model name was wrong for most of the build. I'd written gemini-3.5-flash into agent/incident_agent.py, the README, and the architecture diagram, because it read like the obvious "use the newest thing" choice. When I finally ran the live path against Vertex AI instead of the offline test fake, it came back 404 NOT_FOUND. That model doesn't exist as a publisher model. The fix was mechanical, swap to gemini-2.5-flash, which does exist and is the current Flash tier, but the lesson wasn't: an offline test suite full of mocked classify_fns will happily pass forever against a model name that has never once talked to a real API. I only caught it because I forced myself to run the live deploy path before submission instead of trusting 36 green offline tests.

The sharper one: during a final read-through I found that the shipped validator/corroboration.py had if False: where if not b_failed: should have been, with a comment reading "DEMO-BREAK bypassed for filming." At some point during filming prep I'd disabled the corroboration check to show what an unguarded version looks like on camera, and the disabled version got committed. That one line is the entire thesis of the project. With it disabled, a single flaky region produces a full incident artifact, exactly the false alarm the whole design exists to prevent. The existing test suite does catch it, three tests go red the instant that line ships, which means it got committed without anyone running the tests first. I restored the real check and then wrote tests/test_no_demo_break_shipped.py, which greps the actual shipped source for that marker string and for leftover .demo-break-backup files, verified red with the break in place and green once removed. Remembering not to ship a disabled safety check isn't a control. A test that fails if the marker exists is.

What's actually running right now

The corroboration gate is provably unbypassable in the code, not just documented as a design intent. tests/test_agent_guard.py asserts the agent path is unreachable before the validator passes, and tests/test_verifier_red_green.py does the actual break-then-restore: it disables the validator, confirms the run gets wrongly accepted, then restores it and confirms the run gets rejected again. That's the same category of test that caught the real demo-break bug above, which is a good sign the methodology works rather than just a lucky catch.

It's deployed and running on real GCP infrastructure right now, not a local demo: Cloud Scheduler firing every two minutes, a real Cloud Run Job, a real Cloud Function in a second region, real Firestore claims, a real GCS bucket. 36 tests pass offline with zero network calls and zero credentials, and the live path, real probes, real corroboration, real Gemini 2.5 Flash draft, real GCS write, has been run end to end against that live deployment.

The real lesson

The hardest part of this build was never the model call. Gemini's job here is small and late: classify a failure and write a few sentences, after a plain arithmetic check has already decided the failure is real. The hard part was proving the background execution actually happened without anyone watching, rather than just asserting it did. An empty GCS folder becoming a populated one, with a wall clock visible and the laptop closed, is a stronger claim than any amount of prompt engineering, and it's also a much easier claim to accidentally fake or accidentally break, which is exactly what the demo-break bug did.

The other lesson was about what a clean clone actually contains versus what I thought it contained. My first honest reproducibility test, cloning the repo into an empty directory and following my own README with nothing else on that machine, failed immediately: the README assumed a sibling directory and a virtualenv setup that only ever existed on my dev machine. Everything in the current README is what was actually run, in that order, from a directory that had never seen this project before.

Where the edges still show

Two observers is enough to reject a single flaky region, but it isn't enough to distinguish "my service is actually down" from "us-central1 and europe-west1 both lost the same upstream DNS provider at the same time." A correlated failure across both regions would still corroborate and produce an incident, a real gap, not a hypothetical one, given how much infrastructure a handful of providers actually share. More independent observers, ideally on genuinely different cloud providers, is the next real improvement, and it's a config change to infra/deploy.sh rather than a redesign.

The all-clear path also has a real lag: it only fires on the next tick that happens to see recovery, so there's up to one Scheduler interval between the fix landing and the all-clear file appearing. And there's currently no alerting on the rejected path, a rejected run is written to Firestore for audit but nothing surfaces it, so a founder who wants to see "we noticed and correctly ignored a blip" has to go looking for it. Both are small, both are next.

Built With

• adk
• cloud-run
• cloud-scheduler
• firestore
• google
• google-cloud
• opentelemetry
• python