Codex-app-LoopCrew

A reviewed crew of Codex agents in app, that keeps project state in files, proves every 'done' with evidence, and — in a public A/B on GPT-5.6 — out-hardened a solo session on security and invariants.

Comment

Loop Crew runs a reviewed crew of Codex agents inside the Codex app — and tells you the one moment a human is actually needed.

What it is, and why I built it

Loop Crew is a Codex skill that runs a small crew of specialized Codex agents — "lanes" — on one project, with a built-in review loop, entirely inside the Codex app. No CLI, no terminal, no logs to tail. You describe a goal once; the crew assembles itself, splits the work, checks its own output, and surfaces the single moment a human is genuinely needed.

The honest origin is personal. I came to Codex through the app, not the CLI — it's approachable, quick to start, and where I actually work. For a long time I assumed a session was one thing running in the background; I had no idea a Codex session could open other sessions as real conversations, right inside the app. The first time I watched one agent spin up another as a live thread I could click into, it clicked: a whole team could work without ever leaving the app — and almost no newcomer knows it's even possible.

That gap is what Loop Crew is for. Most people — especially non-developers — meet Codex through the app, not the terminal, so "just spawn a swarm of agents in the CLI" isn't an answer for them; it's the kind of thing that pushes newcomers away. I wanted real multi-agent work reachable inside the app — something you grow into, not something that funnels you out to a CLI you never signed up for. And once a team lives in the app, two problems had to be solved: chat history is disposable — close a conversation and the project's real state goes with it — and "done" was usually just claimed, never proven. So the crew keeps its state in files that outlive any chat, and makes every "done" something an agent must prove.

What it does

Six ideas do the work. A lane is one agent's standing job — product, a build lane, review, and specialists only when they're earned — each owning its files so agents never collide. Work has a status you can read: tasks move through fixed stages recorded in files, so if a chat dies the next session resumes where it stopped. "Done" is proven, not claimed: an agent runs the tests and leaves the results on disk; no readable proof means blocked, full stop. A different agent reviews the work — the builder never signs off on itself, and the reviewer hunts for "looks finished but is wrong." Anything with a screen waits for you: passing tests isn't enough for user-facing work; it sits until a human opens it and confirms. And one dashboard raises a "Ready for you" banner naming the exact conversation to open — so you can ignore the agents until they need you.

How we used Codex

Two ways, both load-bearing.

The product is built on the Codex app. This isn't a wrapper that shells out to a CLI — the crew lives inside the app. The skill calls Codex's own create_thread tool to open each lane as a real Codex conversation, and send_message_to_thread to pass typed handoffs between them. Each lane is auto-seeded with its role, write scope, and model tier, so the team assembles itself with no setup. It extends Codex's "Loop Engineering" — the app's model for durable, resumable, auto-chained sessions — from one agent to a reviewed team.

And the whole thing was built with Codex. The skill, its reference scripts, the demo, and both arms of the A/B study below came from driving Codex. I dogfooded the loop on itself — using the crew to build the crew — which is how I found its sharpest edges.

How we used GPT-5.6

By default, every lane opens on the highest model tier the host offers, at xhigh reasoning — quality first, though a human can dial any single lane down. On the Codex app that top tier is GPT-5.6, so a "crew" is really several premium GPT-5.6 sessions at once; the skill even flags a mismatch if a lane isn't running the tier it recorded. The honest comparison ran on GPT-5.6 too: I built the same app twice on one GPT-5.6 model at xhigh — once through the loop, once as a plain session — so the only variable I measured was the process, not the model.

How I built it

State is files, not chat: goals, the request tracker, constraints, handoffs, the lane registry, per-command evidence, and an append-only transition log all live in the repo under docs/loop/, and every request runs a strict, file-recorded lifecycle. The completion gate is a small local script, completion_gate.py — it reads flat evidence records (request, command, exit code, timestamp) and emits SHIP_CHECK_OK only on passing exit-code evidence; anything missing, malformed, or non-zero fails closed to BLOCKED. It validates records; it never claims to have run the tests itself. On top sit the independent review lane, the human-QA gate (user-facing work stays in REVIEWING until a human_qa: confirmed record lands), and a local dashboard the skill starts itself — a read-only Python viewer on 127.0.0.1. Git is the backbone: one shared branch, lane-labelled commits (CODEX_LANE=<lane> git commit), and a pre-commit scope guard that rejects out-of-scope files and fails closed if CODEX_LANE is unset. All local Python, no cloud.

Does it actually help? (the honest part)

So I ran a public, controlled A/B: the same local expense-analysis web app, built twice from identical requirements and fixtures on the same model (gpt-5.6-sol, xhigh) — once through the loop, once as a plain Codex session with the skill removed (isolation verified). Both codebases are public and were scored by the same rubric, every serious finding independently re-verified.

Dimension Solo session Loop
Correctness on edge input 6 7
Invariant enforcement depth 6 8
Security 7 9
Test quality 7 7
Maintainability 8 8
Average 6.8 7.8

The loop won by about a point, concentrated where a review-and-invariants process should: security and defense-in-depth — DB-enforced invariants, an airtight XSS/CSRF posture, frozen regression matrices. It tied on maintainability and test quality.

But the cost is not subtle. The loop produced roughly 8.5× the code (13,622 vs 1,606 lines) and burned one to two orders of magnitude more time and tokens — about 10 minutes for the solo session versus multiple days for the loop (solo used 733,070 tokens, exact; the loop's is only estimable). And it is not magic: the same review found real bugs in the loop's own output, including one that violates its headline invariant — a PDF path that silently drops lines it doesn't recognize, the exact "no silent data loss" rule the whole skill exists to protect. The honest takeaway is "match the machinery to the stakes," not "always use the loop."

Challenges and what I learned

The hardest problem was philosophical, not technical: making "done" mean something. Early on the crew would happily mark work complete against a check that stays green no matter what you feed it — so I learned to demand red-capable criteria, where every acceptance command can actually fail when the requirement is violated. The second lesson came from dogfooding: independent review plus a human-QA gate caught bugs machine checks never would have (a port collision and a broken pie chart, both mid-run), and yet the loop still shipped that silent-data-loss bug. Verification raises the floor; it doesn't guarantee the ceiling.

When not to use it

Reach for a plain Codex session for a small, low-risk task one agent finishes in a sitting — under two hours — where auditability, handoff recovery, sensitive-data gates, and real parallel lanes don't matter. The skill knows this about itself: during intake, if the job is that small, it recommends not using the loop.

Grounded in prior work

This is a fusion, not a fresh invention. It turns Codex's Loop Engineering and its cross-thread tools into a disciplined team with disjoint write scopes and independent review; the acceptance-and-review discipline was distilled from a survey of 38 community skills (Matt Pocock's collection), where 28 of 38 converged on the same idea. The append-only decision log adapts engram, the Cartridges paper (arXiv:2506.06266), and deepseek's Engram as plain repo-readable files, not trained caches; the dashboard's look comes from companion skill han-design-skill-v1.

Links

Built With

Share this project:

Updates