-
-
RADA — blind model selection with deterministic verification
-
Working gameplay: waves, scoring, core damage and restart
-
NEFARIN concept art — many model paths converging on one verified result
Inspiration
I regularly use several AI coding agents and kept becoming the human router between them: copying context, switching applications and guessing which model would be best for each task.
Every model confidently claimed that it could solve almost anything. That made self-assessment unreliable.
The central question became:
What if the models had to compete anonymously on the quality of their plans, judge one another, and leave an auditable explanation of the final decision?
No individual vendor has a strong incentive to build a neutral router that may conclude that a competing model is better for a particular task. RADA was created as an independent layer above them.
What it does
RADA — Ranked Agent Decision Arena — is a local, vendor-neutral control plane for AI coding agents.
It connects Claude Code, Codex CLI, Gemini CLI and Grok Build in one decision process:
- Bidding: every agent receives the task and submits a structured proposal containing its confidence, approach, risks and estimated effort.
- Blind voting: proposals are anonymized and shuffled. The agents rank the plans without seeing their authors, and RADA calculates the result using Borda count.
- Execution: only the selected winner works in the repository, avoiding the cost and merge conflicts of running every agent independently.
- Verification: a project-defined deterministic command returns
PASS,FAILorINCONCLUSIVE. This result — not an AI opinion — determines the final status. - Independent review: the runner-up reviews the winner’s work but cannot override the verifier.
- Audit: bids, votes, mappings, roles, results, hashes, verification and review are recorded in JSON.
RADA provides three interfaces: a terminal CLI, a shared model room and a browser interface with live bid cards and vote results. It is written entirely with the Python standard library and includes a deterministic mock mode, allowing judges to run the complete process without API keys.
For larger tasks, the optional :sztab mode activates RADA 8S collaboration. The winner remains the only model with write access, the runner-up stays independent for final review, and additional agents contribute read-only testing and UX/red-team advice. This gives the project multiple perspectives without creating concurrent-write chaos.
Inside the arena, the Codex CLI agent (running GPT-5.6) competes under exactly the same rules — it bids, votes, advises, executes and reviews like every other model.
How we built it
The project was developed during Build Week through a tight engineering and verification loop led by Codex running GPT-5.6.
Codex implemented the core fixes as separate, auditable commits: correct reviewer selection, explicit agent failure handling, one-shot command routing, consistent audit records and Windows portability. It later added configuration validation, worker-failure isolation and a deterministic verifier with restricted environment access.
Live runs exposed additional integration problems that mocks could not reveal, including Windows UTF-8 handling, command-line prompt limits and differences in how individual CLIs accept input. These findings produced another tested hardening pass using standard input for long prompts.
RADA then designed its own optional collaboration protocol. Codex implemented :sztab, including:
- deterministic role assignment from the complete vote ranking;
- a single lead with write access;
- an independent runner-up reviewer;
- read-only test-strategy and UX/red-team advisers;
- failure isolation for unavailable advisers;
- hashes proving what information was delivered to the lead;
- references showing which pieces of advice were accepted or rejected.
A live RADA run produced NEFARIN: Core Defense, a self-contained offline browser game. A subsequent staff run used separate lead, adviser and reviewer roles to inspect and verify the artifact. The final repository contains the game, its regression tests and the orchestration that created it.
The public test suite now contains 54 tests, verified on clean Windows and Linux environments.
Codex Session ID: 019f71df-f990-71c2-976b-5020868ba57f
Challenges we ran into
The first challenge was normalizing four unrelated command-line tools. Each uses different flags, output envelopes, authentication flows, permissions and prompt-delivery mechanisms.
Windows revealed another class of problems: Unicode output, PowerShell script policies, .cmd wrappers and command-line length limits. We addressed them without introducing platform-specific behavior into the public configuration.
Agent failures also had to be isolated. One malformed configuration, timeout or unavailable model must not destroy healthy results from the remaining participants.
The hardest design challenge was safe collaboration. Allowing several models to edit simultaneously would create conflicts and destroy accountability. RADA therefore separates thinking from writing: advisers remain read-only, one lead implements, and another model reviews independently.
We also learned to separate three concepts that are often incorrectly combined:
- execution success;
- deterministic verification;
- model-based review.
A reviewer can raise useful concerns even after tests pass, but it cannot turn a deterministic FAIL into success.
Accomplishments that we're proud of
A real four-model run produced the exact situation RADA was designed to detect: the agent with the highest self-confidence finished last, while the least confident proposal won the blind vote unanimously.
RADA also proved that collaboration does not require giving every agent write access. The staff protocol produced attributable advice, recorded the lead’s decisions and preserved reviewer independence.
The project now provides:
- anonymous proposal ranking;
- deterministic and reproducible tie-breaking;
- failure isolation;
- an auditable JSON record;
- a verifier that never silently converts missing evidence into success;
- optional multi-agent collaboration with one accountable writer;
- a working game created and reviewed during live runs;
- mock execution requiring no paid services;
- zero runtime dependencies outside Python’s standard library.
What we learned
Self-confidence is not the same as competence. Models are often optimistic when judging themselves but much more discriminating when comparing anonymous plans.
We also learned that simply adding more models does not automatically create a team. Effective collaboration requires explicit roles, limited permissions, failure isolation and a record of who influenced the final result.
Most importantly, an AI reviewer and a deterministic verifier serve different purposes. The reviewer provides judgment and criticism. The verifier provides evidence.
No AI model gets the final word. A deterministic verifier does.
What's next for RADA — Ranked Agent Decision Arena
Once the workspace is configured, Codex can act as the external control room: the user describes a goal once, while Codex launches RADA, monitors every phase and collects the evidence.
Next, we will benchmark new cloud and local models as leads, reviewers and advisers on repeatable golden tasks, measuring reliability, latency and cost rather than relying on impressions.
The system is not limited to games or programming. The same workflow could guide research, books, films and other complex projects: multiple models propose approaches, divide responsibilities, challenge mistakes and work toward a verifiable result.
Our next creative experiment will ask the team that built NEFARIN to design its trailer, generate controlled video variants using tools such as Dreamina and Seedance, and review the outputs with the same discipline used for code.
The long-term goal is a model-independent operating system for teams of artificial intelligences.
One goal. Many minds. A verified result.
Log in or sign up for Devpost to join the conversation.