agent toolkit

Inspiration

In an earlier agent workflow, plans, progress, and reviews lived in separate Markdown files. They drifted out of sync, agents missed requirements, and useful lessons disappeared after each session.

We built agent-toolkit so a new Codex session can pick up a task with the right context, follow the same delivery process, and leave important decisions to the developer.

What it does

agent-toolkit is an installable Codex plugin for managing the full development lifecycle:

• Turn research into a PRD, technical specification, and roadmap.
• Break milestones into self-contained tasks.
• Implement each task in an isolated Git worktree.
• Open a pull request and drive CI to green.
• Run separate test, review, fix, and verification steps.
• Require evidence for every Definition of Done criterion.
• Keep merging behind a human approval gate.
• Save useful lessons as project rules for future sessions.

Tasks can live in GitHub Issues, Linear, or local files. Pull requests and CI remain the source of truth for review and delivery.

The companion utils plugin adds research, LLM Wiki maintenance, retrospectives, and security scanning.

How we built it

The project is a collection of Codex skills, specialized agents, tracker integrations, and validation scripts.

Git worktrees isolate implementation. Pull requests and CI record reviews and test results. Separate agents handle test writing, code review, and final verification so they do not rely on the implementation session’s context.

Important rules are enforced by scripts, not prompts alone. The project-rule resolver loads instructions from the correct repository and commit. The GitHub lifecycle script validates task labels and state transitions. Repository checks catch invalid manifests, version drift, stale generated agents, and broken lifecycle contracts.

We used Codex with GPT-5.6 Sol and GPT-5.6 Terra to research plugin behavior, design the workflow, implement features, write tests, review pull requests, apply fixes, and verify the result. We also used agent-toolkit to develop agent-toolkit.

Challenges we ran into

Codex sessions do not automatically share context. Each task therefore needs enough information for a new session to work correctly without relying on earlier conversations.

Prompt instructions were not reliable enough for critical operations. Agents could load the wrong project rules, use the wrong repository revision, or update a GitHub issue incorrectly. We moved those operations into tested scripts.

Fork contributions were another challenge. Issues and pull requests belong to the main repository, while contributor branches are pushed to a fork. agent-toolkit validates each repository role so gh commands cannot silently target the wrong repository.

Codex also limits nested agent delegation. We designed dev:auto to coordinate sibling agents while keeping implementation, review, and verification separate.

Accomplishments that we're proud of

agent-toolkit covers the full development lifecycle, from product discovery and planning through implementation, review, verification, and retrospectives.

The repository passes 29 focused tests and 11 repository-wide validation checks. The same checks run in GitHub Actions on every pull request.

It is also used on real projects, not just demos. I use agent-toolkit to manage ongoing development in sekai-kb and lagunabeach-md. Both projects keep their development configuration and reusable rules under .agent-toolkit/.

agent-toolkit also uses its own workflow to manage issues, pull requests, reviews, CI checks, and verification.

What we learned

Project state should live in trackers, commits, pull requests, and CI, not in conversation history.

Each task must contain enough context for a new session to execute it correctly.

Critical rules need deterministic validation. Prompts are useful for guidance, but they should not be the only protection around repository selection, task state, or generated files.

Automation still needs clear authority. agent-toolkit can implement, test, review, fix, and verify work, but merging remains a human decision unless the project owner explicitly enables auto_merge.

Built With

• codex
• gpt-5.6