PocketDev Agent Control

Supervise Codex from anywhere with verified diffs, tests, GPT-5.6 risk reviews, revision checkpoints, and auditable human approval.

Comment

PocketDev Agent Control

Inspiration

AI coding agents can implement meaningful changes quickly, but supervising that work is still surprisingly difficult. Terminal output tells a developer what happened, yet it does not clearly answer the questions that matter before approval:

  • What files actually changed?
  • Did the real tests pass?
  • What risks remain?
  • Did the agent address the reviewer’s feedback?
  • Who approved the final result?

I built PocketDev Agent Control to turn that fragmented process into a structured, human-controlled workflow that can be supervised from anywhere.

PocketDev already connected developers to their coding environments remotely. Build Week gave me the opportunity to extend that foundation into a control plane specifically designed for Codex.

What it does

PocketDev Agent Control manages coding work as a durable task with explicit lifecycle states:

  1. A developer creates a task for an approved repository.
  2. Codex performs a bounded implementation stage.
  3. PocketDev collects changed-file, Git diff, command, and test evidence.
  4. GPT-5.6 reviews the bounded artifacts and produces a structured risk assessment.
  5. The developer can approve, cancel, or request changes.
  6. A requested change starts a new revision stage with the instruction preserved.
  7. The final decision and supporting evidence remain in an ordered audit trail.

The system does not infer repository truth from conversational output. Tests come from the test runner, file changes come from Git, and lifecycle state belongs to the control plane.

It also avoids claiming that code was merged, pushed, or deployed unless that operation actually occurred.

How I built it

I used Codex with GPT-5.6 throughout the project to:

  • Audit the existing PocketDev architecture
  • Identify the strongest Build Week extension
  • Design the task lifecycle and approval checkpoints
  • Implement shared TypeScript contracts and transition rules
  • Add persistent task, event, and approval models
  • Build authenticated GraphQL operations
  • Integrate a bounded Codex worker into the VS Code extension
  • Add reconnection, cancellation, revision, and relay-token handling
  • Write focused tests and security documentation
  • Review implementation risks and refine the submission

The architecture separates responsibilities deliberately:

  • The API owns authentication, durable task state, event ordering, approvals, and reports.
  • The VS Code extension owns workspace validation, Codex execution, Git inspection, and test execution.
  • Codex performs implementation and revision stages.
  • GPT-5.6 provides artifact-grounded risk review.
  • The developer retains authority over revision and final approval.

Judge sandbox

The complete PocketDev platform contains proprietary infrastructure, so I created a deterministic judge sandbox that demonstrates the Agent Control workflow without exposing unrelated product code or requiring production credentials.

Judges can run it with:

npm start

Then open:

http://127.0.0.1:4173

The sandbox requires only Node.js 18 or newer. It includes interactive task progression, ordered events, test evidence, changed files, revision requests, cancellation, and approval.

Sandbox evidence and GPT-5.6 output are visibly labeled as fixtures. The repository also includes the relevant production integration source for technical review.

Tests run with:

npm test

Challenges

The largest challenge was separating trustworthy evidence from agent narration. A model can describe a successful test run, but that description is not equivalent to an observed process result. I designed the workflow so evidence is collected from the underlying repository and test command instead.

Another challenge was cancellation. A task should not become cancelled merely because the API requested cancellation. It first enters cancelling and reaches cancelled only after the worker acknowledges the abort.

Reconnection introduced a similar issue. Live subscriptions are useful for latency, but they cannot be the only source of truth. Worker commands and ordered events therefore need durable storage and replay behavior.

Finally, I had to balance judge access with protection of the existing proprietary PocketDev platform. The resulting repository provides a runnable sandbox, focused implementation source, tests, architecture documentation, and explicit boundaries between production behavior and deterministic fixtures.

What I learned

I learned that supervising an agent is fundamentally a state-management and evidence problem—not simply a chat-interface problem.

The most important design principles were:

  • Treat repository content as untrusted input.
  • Keep execution bounded to an approved workspace.
  • Persist human instructions and decisions.
  • Make retry and reconnection behavior idempotent.
  • Separate observed evidence from AI-generated analysis.
  • Never allow polished UI copy to overstate what actually happened.

Codex accelerated implementation significantly, while GPT-5.6 helped reason through architecture, risk, and review quality. Human judgment remained essential for defining scope, correcting assumptions, and deciding what evidence was sufficient for approval.

What’s next

Next, I would expand Agent Control with:

  • More repository and test-runner adapters
  • Stronger deployment isolation
  • Configurable approval policies
  • Richer artifact comparison between revisions
  • Team review and delegated approvals
  • Additional deterministic end-to-end tests
  • Optional pull-request creation after explicit approval

PocketDev Agent Control demonstrates a practical model for human supervision of coding agents: Codex implements, GPT-5.6 reviews the evidence, and the developer remains in control.

Built With

Share this project:

Updates