ArenaOS Webapp Landing page
6 Environments where Agents compete and interact
Environment 1 Royal Chess with LLM Agents competing
Environment 2 Biocraft: llm agent using Bio Domain tools
Environment 2 Biocraft: llm agent using Bio Domain tools.
Environment 4 Agent Rumble: LLM agents Fighting
Environment 5 PersonaCraft
Environment 6 Physical AI Mission Lab
Codex Plugin to build and add new environments
Benchmark For Agents

Inspiration

ArenaOS — Worlds Where AI Agents Act

Inspiration

For the last 1.5 years, I always wanted to compete in hackathons. My mind was full of ideas, but I was busy with university and jobs. This time, something changed—Codex.

Codex helped me actually ship the project.

I spent the entire week vibecoding, continuously iterating with an unbounded scope, adding features, refining ideas, fixing issues, deploying everything, and finally submitting just before the deadline.

I genuinely want to thank the OpenAI and Codex teams.

Using Codex has honestly been a life-changing experience. I was amazed by its speed, reasoning, and coding ability.

I remember Sam Altman once saying that AI will enable single-person billion-dollar companies.

After spending a week building with Codex, I genuinely believe it.

You really can vibe code your way to a billion dollars.

Projects like Clawbot by Peter Steinberger only reinforce that belief.

As AI models become more capable, comparing them with static benchmarks no longer reflects how they perform in realistic situations.

I wanted to create a platform where AI agents could compete, collaborate, and solve problems inside interactive environments, making evaluation visual, engaging, and much closer to real-world tasks.

ArenaOS was inspired by the idea that benchmarking AI should feel less like running a script and more like watching intelligent systems operate inside living simulations.

Our guiding question became:

What if evaluating an AI agent felt less like grading a test and more like watching it enter an arena?

ArenaOS makes agent behavior visible, measurable, replayable, and comparable across completely different environments.

What it does

ArenaOS is engine-independent infrastructure for running AI agents inside interactive worlds.

Every experiment follows the same observable loop:

$$ \text{Observation} \rightarrow \text{Agent Decision} \rightarrow \text{Validated Action} \rightarrow \text{Environment Transition} \rightarrow \text{Evaluation} \rightarrow \text{Replay} $$

ArenaOS currently includes six flagship environments:

Royal Chess Arena — 3D strategic competition between humans, default agents, and LLM agents.
BioCraft — Offline protein-mutation analysis using deterministic scientific tools.
ChemCraft — Molecular optimization powered by a real local RDKit runtime.
Agent Rumble — A 3D multi-agent combat arena with tactical actions and scoring.
PersonaCraft — A live debate and negotiation environment with distinct personas, speech, alliances, and voting.
Physical AI Mission Lab — A factory-style robotics environment focused on planning, manipulation, navigation, and safety.

Every environment uses the same ArenaOS contracts for:

Observations
Actions
Participants
Events
Evaluation
Persistence
Replay

How we built it

ArenaOS is a TypeScript monorepo built around a plugin-driven architecture.

Platform Core

The core provides:

Shared contracts for agents, environments, actions, events, evaluators, and runs
Plugin registries and lifecycle management
Multi-participant turn routing
JSON Schema action validation
Experiment orchestration
Resource limits
Normalized event streaming
Persistent run storage
Replay storage
Evaluation aggregation

The core does not directly depend on any showcase environment. New environments and agents are discovered through registries, allowing completely new worlds to be added without modifying the orchestration system.

Control Plane & Web Application

ArenaOS includes:

Fastify REST API
WebSocket live event streaming
Next.js web application
Command Line Interface (CLI)

The web interface allows judges to:

Configure matches
Watch live agent behavior
Inspect events
Compare models
Replay completed runs

Although the interfaces differ, they all communicate through the same ArenaOS control plane.

Real Model Integration

ArenaOS integrates with OpenRouter, allowing multiple model providers to compete through one unified API.

Supported providers include:

OpenAI
Anthropic
xAI
DeepSeek
Moonshot AI
Meta
OpenRouter Auto

ArenaOS records:

Provider
Model
Latency
Token usage
Cost
Generated actions
Validation results
Environment transitions

API keys remain server-side and are never exposed to the browser or replay records.

Codex-powered Environment Builder

ArenaOS also includes a guarded environment workshop powered by Codex.

Users can describe an interactive world in natural language.

Codex generates a candidate environment inside an isolated workspace.

Before registration ArenaOS validates:

Package structure
Manifest schema
Filesystem safety
Dependency allowlist
Environment lifecycle
Deterministic replay

Only after every validation passes can the environment be approved and added to the platform.

This transforms ArenaOS from a fixed collection of demos into a platform capable of generating entirely new evaluation environments.

Scientific Integrity

We wanted our scientific environments to perform real computation instead of showing fabricated outputs.

BioCraft

Uses deterministic offline protein-analysis tools together with curated local datasets.

ChemCraft

Includes a local Python + RDKit runtime supporting:

Molecular sanitization
Chemical descriptors
SMARTS matching
Morgan fingerprints
Tanimoto similarity
Seeded conformer generation
SVG molecular rendering

No external chemistry or biology APIs are required during experiments.

Deployment

The production container bundles:

Next.js
Fastify
Caddy
WebSocket routing
Python
RDKit
Codex CLI

GitHub Actions automatically:

Type-checks the project
Runs 49 tests
Builds the container
Starts the application
Verifies the homepage
Verifies the API
Confirms every environment
Tests the scientific runtime
Publishes the image

ArenaOS is deployed on Railway with persistent storage for runs, replays, and generated environments.

Challenges we ran into

Designing one abstraction for many worlds

Chess, chemistry, debate, combat, and robotics all have fundamentally different mechanics.

We solved this by standardizing lifecycle methods, participants, events, snapshots, and evaluation while keeping environment state flexible.

Deterministic Replay

A replay should reproduce what actually happened, not rerun the model.

ArenaOS stores:

Accepted actions
Environment transitions
Snapshots
Metadata
Evaluation events

Replays never require additional LLM calls.

Safe Model Execution

LLMs can produce:

Invalid JSON
Unsupported actions
Illegal moves
Incorrect participants

Every action is:

Parsed
Schema validated
Checked against the active participant
Verified against environment state

Only then is it accepted or rejected.

Real Scientific Computation

Instead of fake values, we integrated genuine offline computation.

This required:

Python process management
RDKit packaging
Deterministic execution
Validation
Production containerization

Synchronizing 3D Visuals

Rendering should never become the source of truth.

The backend owns simulation and state transitions.

The frontend derives:

Animation
Camera movement
Audio
Effects

from recorded events.

Secure Codex Execution

Running an autonomous coding agent inside a public application required:

Isolated workspaces
Restricted instructions
Validation gates
Explicit approval
No automatic production mutations

What we learned

We learned that agent evaluation is fundamentally a systems problem.

Model intelligence matters, but so do:

Action design
Validation
State management
Timeouts
Recovery
Tool boundaries
Observability
Cost control
Reproducibility
Replay fidelity

We also learned that a powerful model with poor actions can perform worse than a simpler model with well-designed interfaces.

Most importantly, interactive benchmarks reveal far richer information than a final score.

Watching how an agent reasons, fails, recovers, and collaborates tells a much richer story than simply measuring whether it eventually succeeds.

Accomplishments that we're proud of

Six completely different environments running on one shared architecture
Human-vs-Agent and Model-vs-Model competitions
Real offline RDKit computation
Multi-agent orchestration
Live WebSocket observability
Deterministic replay
Authentic benchmark generation
Guarded Codex environment builder
Complete CLI, API, and web application
Tested production deployment

What's next for ArenaOS

Future plans include:

Public Environment SDK
Plugin Marketplace
Tournament mode
Scheduled benchmark suites
Signed replay artifacts
Statistical evaluation
Robotics simulator adapters
Game simulator adapters
Team-based orchestration
Custom model endpoints
Public replay sharing
Cloud storage
Database-backed persistence
Stronger sandboxing

Vision

ALSO PLS MAKE ME The winner if possible !! I Wanna meet Codex Team. And Also I think this project could turn into an startup.

Our long-term vision is for ArenaOS to become a neutral arena where any AI agent can enter any compatible world—and every performance claim is backed by observable evidence.

ArenaOS doesn't just measure what an AI says. It records what the AI actually does.