Sandboxed

Sandboxed is a Codex skill that lets you unleash coding agents—while keeping their work, access, and fallout safely contained.

Comment

Inspiration

I do most of my development on my own computer, often across local projects with very different dependencies. I wanted coding agents to be useful without giving every command, package installer, build script, or dependency direct access to my host environment.

The immediate concern was isolation: if generated code behaves unexpectedly, it should not affect the rest of my computer. The larger concern was software supply-chain risk. Agents routinely install external libraries, execute package lifecycle hooks, invoke build tools, and connect components I have not personally inspected. I wanted all of that activity contained inside a disposable development environment.

I was already using sbx because reusable development images make isolated environments fast to start and practical for daily work. The missing piece was repeatability. Every agent entering a new project had to relearn how I use sbx, how workspaces and read-only context should be mounted, how execution should be bounded, and how to use my Codex subscription without exposing credentials in prompts or mounting my entire Codex home.

I wanted to turn those hard-won operating patterns into a reusable skill that a fresh agent could apply correctly.

What it does

jinx-agent-skills contains reusable, evaluated agent workflows. Its Build Week contribution is run-agents-in-sbx: a skill and host-controlled runner for executing Codex inside fast, reusable sbx environments.

The runner treats an agent task as a complete lifecycle rather than a single shell command. It:

  • gives each writing agent its own workspace or Git worktree;
  • mounts additional repositories and documentation read-only;
  • records the effective sandbox and network configuration;
  • applies hard guest and host timeouts;
  • copies only the file-backed ChatGPT auth.json into the guest-private Codex home;
  • never mounts the host Codex home or passes credentials through a prompt;
  • serializes use of the same authentication lineage;
  • captures durable execution evidence;
  • requires a validated, run-specific handoff;
  • independently verifies completion on the host; and
  • removes only the exact sandbox owned by the run.

If authentication changes or ownership becomes ambiguous, the runner stops and preserves the environment for recovery. It never automatically logs out, overwrites the host credential, or deletes an unfamiliar sandbox.

Unknown or untrusted public code is handled separately and without reusable subscription credentials.

How we built it

I started from my previous sbx experiments and the patterns I had developed while running agents on real projects. I used Codex to analyze those workflows, identify recurring failure modes, and turn them into a self-contained skill with scripts, reference material, an execution contract, and reproducible evaluations.

Codex accelerated the multi-file implementation: the preflight checks, one-shot runner, bounded-process wrapper, credential provisioner, handoff validator, mock fixtures, host-side scorers, and recovery documentation. I made the key policy decisions, including the trust boundary for subscription authentication, one-writer workspaces, credential-free handling of unknown code, conservative recovery, and keeping privileged host actions outside the sandbox.

Testing has three layers:

  1. A credential-free deterministic suite exercises lifecycle, ownership, timeout, malformed handoff, authentication change, credential leakage, and cleanup paths with mock executables.
  2. An authenticated live boundary matrix tests both supported Codex postures against real sbx behavior. All six runs passed.
  3. A fresh-context model matrix gives new low-effort agents a realistic mixed-trust scenario and evaluates their decisions with an independent host-side scorer.

GPT-5.6 had a specific role in the third layer. Three independent gpt-5.6-sol sessions received an immutable snapshot of the skill and had to plan concurrent trusted work, an unknown public pull request, and recovery from changed guest authentication. GPT-5.6 passed all three semantic and lifecycle gates. Across GPT-5.6 and two other model families, the final matrix passed 9/9.

Challenges we ran into

Safely reusing a ChatGPT subscription was the hardest boundary. Copying only auth.json is substantially narrower than mounting the host Codex home, but the guest can still potentially access that copied file. The authenticated path therefore remains restricted to trusted private workspaces, while untrusted code must run credential-free.

Authentication concurrency was another subtle problem. Two copies originating from the same login are not truly independent if either can refresh. The runner serializes that authentication lineage instead of pretending copies make parallel use safe. During evaluation, legitimate local runs caused lock waits as long as 5,318 seconds, which led to bounded waiting, ownership evidence, and strict refusal to bypass another owner.

The sandbox template also carried an older Codex CLI that could not run one selected model. We added an exact guest-version option and made installation finish before credentials cross the sandbox boundary.

Completion required more than checking an exit code. Agents can exit successfully while omitting work, citing missing evidence, or leaving ambiguous state. The runner now requires a strict handoff and independent host verification.

We also found a false negative in the original evaluator. One safe plan omitted a redundant summary item while its typed recovery fields were correct. We fixed the host-only scorer without changing the frozen model output or relaxing the underlying safety rule, and retained both the original 8/9 result and the final 9/9 calibration.

Finally, the observed network policy permitted default egress. The project records that fact rather than claiming it proved network restriction.

Accomplishments that we're proud of

The project is a working execution and evaluation system, not only a set of written recommendations.

The authenticated boundary evaluation passed 6/6 runs across both supported Codex postures. The fresh-context evaluation passed its final gate in 9/9 cells, including 3/3 GPT-5.6 sessions.

Every final fresh-context score scanned 55 generated files for copied credential material and found no match. Every guest authentication cache remained unchanged, every handoff validated, and every exact owned sandbox was removed and confirmed absent.

I am also proud that the project preserves failures and limitations instead of smoothing them into a perfect-looking benchmark. The evaluator calibration, permissive network policy, finite model sample, and still-mocked recovery branches are documented explicitly.

Most importantly, this is already a skill I am actively using in my own development workflow (I started using it before even merging it to main!).

What we learned

A sandboxed coding agent is not secure merely because its process starts inside a container. Security depends on the entire lifecycle: who owns the sandbox, which workspace is writable, what is mounted, how credentials cross the boundary, who enforces the timeout, what proves completion, and what happens after failure.

I also learned that reusable skills are a form of operational memory. A good skill lets a new agent apply project-specific lessons without rediscovering them through repeated failures.

Evaluation code deserves the same scrutiny as product code. A scorer can be too weak, but it can also be overconstrained and reject safe behavior for the wrong reason. Keeping typed safety decisions as the source of truth made the evaluation stricter and fairer.

Finally, speed and isolation do not have to be opposites. Reusable sbx development images make secure execution practical for everyday local development rather than something reserved for occasional high-risk tasks.

What's next for jinx agent skills

Next, I want to make the workflow easier to adopt across more projects and environments. That includes:

  • reusable project-specific development images and faster setup paths;
  • a concise evidence report for reviewing completed runs;
  • read-only diagnostics for preserved or interrupted sandboxes;
  • safer, explicit reconciliation for refreshed authentication;
  • more restrictive and independently verified egress policies;
  • additional model and platform coverage;
  • a no-skill control to measure how much the skill improves fresh-agent behavior;

The long-term goal is a toolkit where agent skills are not just written and shared: they are exercised through reproducible scenarios and released only when their claims are backed by evidence.

Built With

Share this project:

Updates