Rulvar

Give one difficult goal to a team of AI agents without provider lock-in or a hosted control plane. Rulvar keeps models, budgets, tools, state and replay in your TypeScript app.

Comment

Inspiration

I wanted to build my own version of an ultracode-style mode. What interested me was not the name or simply giving one model more reasoning time, it was the workflow - give the system one difficult goal, let a strong orchestrator break it into smaller jobs, send those jobs to specialized agents, compare their results, revise the plan when something fails and keep working until the result is actually verified.

But I did not want that behavior to belong to one model, one provider, one coding client or one hosted control plane. I wanted to decide which model should plan, which models should execute and which model should verify the result. I wanted to be able to use GPT-5.6 Sol for orchestration, Terra for implementation, another provider for review and a local model for routine extraction.

I also wanted control to mean more than having a few settings in a user interface. The application should own the provider credentials, tools, permissions, budget, durable state, retry policy, recovery process and execution history.

That idea became Rulvar - my attempt to separate the orchestration pattern behind an ultracode-style experience from any particular vendor and turn it into a normal TypeScript runtime that developers can embed, inspect and control.

What it does

Rulvar is not another coding-assistant interface. It is the runtime underneath that kind of multi-agent experience.

There are three ways to decide what runs next:

  1. I can write the workflow myself as an ordinary async TypeScript function.
  2. A planner model can produce a typed execution script before work begins.
  3. A dynamic orchestrator can decide live which agents to start, what work to delegate, and when the task is complete.

All three modes use the same agent runtime, journal, budget system, tools, events and replay mechanism.

Models are not hard-coded into the workflow, each role can be routed independently. The planner, orchestrator, worker loop, structured extraction, final synthesis and summarization steps can all use different models or providers.

For example - one run can use GPT-5.6 Sol for planning, GPT-5.6 Terra for execution, local vLLM model for review, and a local Ollama model for inexpensive classification. Rulvar also supports OpenAI-compatible gateways and models exposed through the Vercel AI SDK.

The host application remains responsible for the policy:

  • which providers and models are available;
  • which model is allowed to perform each role;
  • which tools an agent may call;
  • how much the complete run may spend;
  • where journals and transcripts are stored;
  • which retries and fallbacks are permitted;
  • how the workflow is resumed after interruption.

Every completed effect is written to a durable content-addressed journal. If the process crashes, is redeployed, or resumes in another Node process, completed model requests and tool executions can be replayed instead of performed and paid for again.

Rulvar does not require a mandatory hosted server or external control plane. The workflow runs inside the application and the application owns its execution history.

How we built it

Rulvar had an existing foundation before OpenAI Build Week. During the event, I focused on making the provider-independent orchestration idea work under real conditions rather than only in a demo.

The main Build Week work included first-class GPT-5.6 support, role-based model routing, accurate provider usage accounting, stronger budget enforcement, durable recovery, deterministic replay and live per-agent observability.

I used Codex with GPT-5.6 Sol as an adversarial engineering partner. I did not ask it to generate one large implementation and assume the result was correct. Instead, I used a repeated review, repair and verification cycle.

For each cycle, Codex:

  1. pulled the latest Rulvar release;
  2. inspected the implementation and documentation;
  3. ran the build, lint, typecheck, packaging, compatibility and test matrix;
  4. designed live OpenAI end-to-end scenarios under a strict $10 testing allowance;
  5. used GPT-5.6 Sol as the orchestrator and planner;
  6. used Luna and later Terra as worker models;
  7. inspected actual provider usage, cost reports, journals, transcripts, checkpoints and replay decisions;
  8. turned failures into reproducible engineering specifications and acceptance tests.

I then reviewed the evidence, chose the implementation approach, applied or supervised the fix, released a new version and used Codex to test the release again.

This process repeated across several releases. Some of the most useful findings were not obvious feature bugs. They were violations of the control guarantees Rulvar was supposed to provide.

For example - the review found cases where cache-write tokens could be counted twice, failed provider responses could lose paid usage, invalid telemetry could corrupt the budget ledger and a legacy journal could report an incorrect historical cost. Later reviews found that provider-controlled text could inject terminal control sequences into progress output.

Each finding forced me to define what "controlled" should mean in practice and I decided that:

  • an unknown model price must remain explicitly unpriced instead of being guessed;
  • a run-level budget cannot be raised after execution begins;
  • provider usage must be validated before it can affect the budget ledger;
  • historical journals should be audited rather than silently rewritten;
  • terminal sanitization belongs at the rendering boundary so it does not alter durable identity or replay behavior.

GPT-5.6 was used in two places during this work: Inside Rulvar, Sol performed orchestration and planning while Terra and Luna handled execution roles. During development, Codex with GPT-5.6 Sol designed and ran the adversarial verification cycles that exposed weaknesses in the implementation.

Challenges we ran into

The hardest part of provider independence was learning that it cannot be implemented as a single adapter interface. Providers represent tool calls, reasoning items, structured output, errors, caching and token usage differently. A workflow may even change providers between its worker loop, extraction step and final synthesis. Rulvar therefore needs one canonical execution history and must reconstruct the correct provider-specific view before every outgoing request. Otherwise, changing models halfway through an agent run would produce an invalid conversation or break replay.

Cost accounting was another major challenge. A budget cannot be reliable if every provider reports usage differently. Cache reads and cache writes may be subsets of the total input count. A failed response may still be billable. Several agents can also be in flight concurrently when a spending boundary is reached. This meant that budget control had to be implemented as a runtime invariant, not as a number displayed after execution.

Durability introduced a different problem. Rulvar must recognize completed work after a crash or deployment, even if unrelated workflow steps were reordered. At the same time, a genuinely different model call must never reuse an old result accidentally.

The final challenge was observability. Logs and progress views are useful only if untrusted provider output cannot manipulate the terminal, clear previous output, or forge CI messages.

These details are less visible than the agent demo itself, but they determine whether the system is actually under the developer's control.

Accomplishments that we're proud of

The most important result is that Rulvar can provide an ultracode-style orchestration pattern without requiring every role to use the same provider.

A single workflow can:

  • use one provider for orchestration;
  • use another for execution;
  • send structured extraction to a cheaper model;
  • use a local model for routine work;
  • fall back to another model after a transport failure;
  • preserve one durable execution history across those boundaries.

The live OpenAI verification covered:

  • dynamic orchestration;
  • planner-generated execution;
  • plan revision;
  • parallel workers;
  • tools;
  • schema-validated structured output;
  • prompt caching;
  • all six invocation roles;
  • external suspension and resolution;
  • resume in a separate Node process;
  • evals and model-judge flows;
  • canary checks;
  • root-budget denial;
  • same-process and cross-process replay.

In the replay test, completed work was restored with zero repeated provider requests and zero repeated tool executions. At the final deep verification checkpoint - more than 1400 tests passed with no failures across Node 22, 24 and 26.

What we learned

The main lesson was that "provider-agnostic" is not a marketing label. It must be preserved at every boundary and that affects everything around: model capabilities, tool formats, structured output, reasoning history, retries, errors, caching, token accounting, pricing and replay identity.

I also learned that "full control" does not mean exposing a large configuration object. Control means that the rules remain true when the process crashes, a provider fails, several agents run concurrently, telemetry is malformed or a model behaves differently from what was expected.

The ultracode-style experience is therefore not only a model feature, it is an orchestration pattern supported by a runtime.

Finally, Codex was most valuable when I used it as a critical reviewer rather than a one-shot code generator. The strongest improvements came from the repeated cycle of testing a real release, finding a concrete counterexample, making a deliberate engineering decision and verifying the next release again.

What's next for Rulvar

The next major step is to build a ready-to-use developer mode on top of the runtime. Developer should be able to give Rulvar a repository and one high-level goal. Rulvar would then plan the work, inspect different parts of the codebase in parallel, assign implementation and review roles, run tests, revise failed approaches and stop only when the configured evidence says the task is complete or the budget is exhausted. The important difference is that the developer would still choose the providers and models behind every role.

The same mode could use only OpenAI models, combine OpenAI and local LLM, route inexpensive work to a local model, or run entirely through a private compatible gateway. The workflow policy and execution history would remain inside the developer's environment.

I also plan to add:

  • a visual inspector for agents, journals, cost attribution and replay decisions;
  • a zero-setup browser or Codespaces sandbox;
  • additional provider and storage conformance suites;
  • more built-in verification and coding workflow patterns;
  • clearer production deployment and migration guides.

The long-term goal is to make powerful autonomous multi-agent execution available as an application-owned capability rather than a feature that exists only inside one provider's product.

Built With

Share this project:

Updates