nokto-agent-orchestrator

Inspiration

We run a real company (Nokto) on agent-written code. Every agent demo ends the same way: impressive output, then a human quietly re-reading every line, because the trust is vibes-based. We wanted trust to be structural — contracts, isolation, adversarial review — so agents can ship real changes to real repositories.

What it does

nokto-agent-orchestrator turns a code change into a YAML task contract: the goal, allowed and disallowed paths, acceptance criteria, the task's own test commands, and a retry budget. The orchestrator then drives a ten-step delivery pipeline:

1. Claude plans with read-only access
2. The plan passes static validation — scope, hard-blocked areas, secrets
3. Work happens in an isolated git worktree — agents never touch the main working copy
4. Codex implements, with Claude as fallback
5. Claude reviews only the diff — never the implementer's own summary
6. Gemini or Codex cross-reviews the change: two frontier models checking each other
7. The contract's test commands run through an explicit command allowlist
8. Findings and failures return to the implementer as concrete correction items
9. Re-review and re-verification, bounded by maxRetries
10. The change ships as a pull request with the complete report. A human merges, always.

How we built it

TypeScript on Node 20. The claude and codex CLIs run as subprocesses with argument-based execution — no shell:true anywhere, so command injection is prevented structurally. Gemini participates as a reviewer through the official Google GenAI SDK (gemini-3.5-flash), and the public project endpoint runs on Google Cloud Run. Phases exchange strict JSON; model output that cannot be parsed is an error, never a guess. State is written atomically after every step, so an interrupted run resumes from the exact last completed step, and every event goes to an append-only JSONL audit log with secrets redacted.

Security is allowlists all the way down: binaries, git subcommands (reset, clean, filter-branch, filter-repo are always blocked; force-push is rejected), and paths. nokto-agent doctor verifies the binaries that actually exist instead of assuming a provider is available.

Challenges we ran into

Fail-closed is a discipline, not a feature. When the secondary reviewer's binary is missing, the run fails with a blocker instead of quietly skipping the check — designing every failure path to stop rather than degrade took as much work as the happy path.

Accomplishments that we're proud of

It runs in production at Nokto. Frontier models adversarially check each other's work. Ten security controls, resumable runs, and full auditability — an agent pipeline you can show an auditor, not just a demo audience.

What we learned

An agentic pipeline is a supply chain: every hand-off between model and machine is an attack surface and a failure point. The controls belong in the structure — allowlists, isolation, atomic state — not in the prompt.

What's next

Gemini as an implementer, the MCP server so other agents can submit task contracts programmatically, parallel fan-out of independent tasks, and per-repo quality profiles for plan validation.

Built With

• claude-code
• cloud-run
• gemini
• git
• github
• mcp
• node.js
• openai-codex
• pnpm
• typescript
• vitest
• yaml