Watcher

Trustworthy, human-governed automated handoffs between desktop AI agents.

Comment

Inspiration

For nearly 8 weeks, I've been building an external app for a commercially available flight combat simulator. The vanilla game commander is very basic, and all attempts at creating a true dynamic campaign and intelligent AI commander have been unfruitful. So, in my efforts to construct a true intelligent AI commander, coupled with an AI driven mission designer, I've been using ChatGPT as the project director, and Codex as the programmer. This has turned into nearly 2000 'sprints' so far. By 'sprint', I mean a single Codex instruction, work, and report.

Since there is no direct pipeline between the ChatGPT browser and Codex, this process initially meant I had to copy/paste instructions and resulting reports between them. This became very tedious, time consuming as well as time lost for sleeping and real life. I needed a way to automate, without subscribing to an API key.

ChatGPT can reason at a high level, and Codex can build, but the workflow often stops when either one finishes. A human still has to notice the result, copy it, switch applications, find the correct conversation, paste it, wait, and carry the next response back. The human becomes the fragile bridge between otherwise capable systems.

Watcher began as an attempt to remove that repetitive relay. Operational experience revealed the deeper challenge: moving text is easy; proving that the text is the exact, current, authorized instruction is hard. A genuine assistant message can still be stale, replayed, associated with the wrong wake, taken from a sibling conversation branch, or delivered to the wrong destination.

Watcher turns those lessons into a local, observable orchestration product.

What It Does

Watcher automates the lifecycle of a desktop-agent handoff, replacing repetitive human copy/paste relay while keeping policy and evidence visible:

  1. Detect that an upstream task or report is ready.
  2. Wake the configured Director when policy permits.
  3. Capture one identified response message.
  4. Validate exact wake-to-response parentage, ancestry, current-path membership, and current-node consistency.
  5. Bind the exact instruction bytes to signed provenance, destination, expiry, and replay state.
  6. Deliver only after policy acceptance.
  7. Record a human-readable activity and evidence trail.
  8. Reject stale, ambiguous, off-path, modified, replayed, or misdirected work.
  9. Watcher coordinates ChatGPT and Codex to send/retrieve instructions and reports to Github, where Github acts as the mailbox exchange between them. In cases where Github uploads may become unavailable to ChatGPT instructions (primarily due to OpenAI safety checks due to wording of 'terms' associated with a combat simulator), then Watcher can scrape text directly from Edge Browser UI, copy it, then paste directly to Codex.

The packaged Preview is deliberately safe and inspectable. Its supported judge path uses bundled synthetic fixtures and a non-actionable in-memory test sink. Judges can observe a valid current-path transaction being accepted once, the identical transaction being rejected as a replay, and a genuine sibling-branch response being rejected before signing or delivery.

How We Built It

Watcher is a local Windows application built with .NET 8 and WinForms. Its architecture separates four concerns that are often conflated in agent automation:

  • Source: where a report or instruction came from.
  • Authorization: whether that exact instruction is permitted now.
  • Delivery: which destination may receive it and by what adapter.
  • Audit: what evidence explains every acceptance or rejection.

The authorization pipeline verifies authenticated message objects rather than trusting whole-page text extraction. It checks direct parentage, full ancestry, current-path membership, current-node consistency, capture method, envelope integrity, destination binding, signing identity, expiry, nonce uniqueness, and durable replay state. Branch divergence and uncertain provenance fail closed.

The release is profile-driven and manual by default. The public Preview isolates its demonstration path from live accounts and external workloads. Experimental browser and local IPC adapters are present for continued engineering, but they are unsupported, explicitly opt-in, and excluded from the judge path.

How GPT-5.6 And Codex Were Used

GPT-5.6 directed the product and safety architecture, decomposed requirements, analyzed forensic lessons from real orchestration failures, and separated authenticity, lineage, integrity, authorization, replay protection, delivery, and audit into explicit contracts.

Codex worker threads implemented and refactored the C# application, created the synthetic demo and test sink, added provenance and replay enforcement, built regression suites, improved the WinForms product experience, produced reproducible packaging, sanitized the public source stage, and generated release evidence.

Human direction supplied the original problem, product goals, operating constraints, incident review, usability feedback, and release authorization. Implementation workers did not self-certify the final candidate.

Codex accelerated the transition from reviewed requirements and observed failures into bounded code changes, executable tests, packaging, and documentation. No unsupported time-saving or productivity metric is claimed.

Build Week Extension

Watcher existed before Build Week. During the July 13-21 submission period it was meaningfully extended with:

  • authenticated current-path lineage validation;
  • signed provenance and destination binding;
  • durable replay and transaction protection;
  • cancellation and interrupted-command hardening;
  • installation trust and release attestation;
  • profile-driven authorization and workflow UI;
  • an isolated synthetic demonstration path;
  • a non-actionable local test sink;
  • reproducible Windows packaging;
  • public-source sanitization and privacy review;
  • judge-facing documentation, release evidence, and no-rebuild setup.

The dated repository history and supplied Codex Session ID document this work.

Challenges

The largest challenge was recognizing that message authenticity is not the same as authorization. Early automation could correctly identify a real assistant message while still selecting an off-path or stale response. That failure mode is subtle because every individual component appears valid.

The solution required binding the entire transaction: conversation, wake token, wake message, assistant response, parent relationship, ancestry, current node, exact envelope bytes, destination thread, signer, expiry, nonce, and replay disposition. It also required making rejection understandable to the operator rather than silently recovering.

Packaging presented a second challenge. The judge path had to demonstrate real orchestration logic without requiring private accounts, API credentials, mutable repositories, or live external services. The synthetic fixture and test-sink architecture preserves the important safety behavior while keeping evaluation deterministic and local.

Accomplishments

  • A runnable Windows product, not only a protocol or proof of concept.
  • Exact wake-to-response and current-branch validation.
  • Signed, destination-bound instruction provenance.
  • Durable replay rejection and duplicate suppression.
  • Human-visible policy, workflow, activity, evidence, and diagnostics surfaces.
  • A self-contained judge package requiring no rebuild or API key.
  • 203/203 packaged Preview offline application checks passed.
  • The 17/17 command-cancellation subset is included within those 203 checks.
  • A separate Stage 3 provenance and intake suite passed 295/295 checks.
  • Zero duplicate acceptances, unauthorized deliveries, or silent recoveries in the published evidence.

These results support the packaged synthetic path and regression claims. They do not claim unattended live-production reliability for experimental adapters.

What We Learned

Agent workflows need transaction security, not merely message transport. The useful unit of trust is not a text block or even an authenticated assistant message; it is a complete, current, destination-bound transaction with inspectable provenance.

We also learned that fail-closed behavior must be legible. A safe rejection that leaves the user guessing becomes an operational failure. Watcher therefore treats activity history, evidence, diagnostics, and human confirmation as product features rather than debug afterthoughts.

What's Next

The next phase is broader independent testing of the Preview, continued GUI refinement, signed Windows distribution, adapter compatibility testing, and carefully bounded manual pilots. Automatic operation will remain opt-in and policy constrained. Stable public provider APIs remain the preferred integration path for server-scale and multi-tenant deployments.

Judge Quick Start

  1. Open the v0.1.0-preview.3 release.
  2. Download WatcherPreview-win-x64.zip and the release summary.
  3. Verify the ZIP SHA-256 against WatcherPreview-win-x64.release-summary.json.
  4. Extract the ZIP and verify package files against manifest.json.
  5. Run RUN-WATCHER-PREVIEW.cmd.
  6. Run the synthetic current-path fixture.
  7. Replay it and confirm rejection without a second acceptance.
  8. Run the synthetic sibling-branch fixture and confirm rejection before delivery.

No rebuild, private account, API key, repository connection, or external sample data is required.

Important Preview Boundary

Watcher Preview is an unsigned Windows Preview, not a production service. Windows may display an unknown-publisher warning. Verify the published hashes before running it. The supported demonstration uses synthetic fixtures and a non-actionable test sink. Experimental live adapters are unsupported and excluded from the judge path. Watcher does not bypass subscriptions, authentication, authorization, usage limits, safety systems, or product policies.

Built With

Share this project:

Updates