arbitr

Inspiration

Cloud compute is often purchased when a job arrives even when the task could wait. Training runs and other batch workloads may have flexible deadlines, but deciding when to run them becomes difficult when prices, hardware requirements and limited capacity all play a part in the feasibility.

We built Arbitr as a solution to this problem. We provide companies with an autonomous system that can ingest its workloads and deadlines, and let it decide when and where to buy compute.

What it does

Arbitr is a buy-side compute scheduling engine. It combines price forecasts with workload constraints to decide whether to run, wait, route to another compatible pool, or report that a job is infeasible.

The interface follows each decision from beginning to end:

Workload -> Decision -> Outcome -> Comparison

Users can inspect a job’s requirements, understand the scheduler’s plan, and compare its outcome with FIFO and cheapest-now scheduling. Historical replay uses observed prices with explicitly simulated workloads and capacity. A separate Docker integration demonstrates local execution, including logs and exit codes.

How we built it

We built the backend with Python, FastAPI, Pydantic and SQLite, and the frontend with Next.js, TypeScript and Tailwind CSS.

We trained our LightGBM forecasting models on historical AWS Spot price data, using chronological splits to evaluate predictions against later observations. The models use recent price history to predict compute prices over a one-to-six-hour horizon.

The application also integrates with the AWS API to retrieve current Spot prices when AWS credentials are configured. These observations provide updated market inputs for forecasting and scheduling.

A rolling-horizon optimiser combines forecasts with job deadlines, hardware compatibility, budgets and shared capacity. It repeatedly updates its plans as new observations arrive.

The scheduling objective is to minimise total expected expenditure while respecting operational constraints:

Total expected expenditure = compute cost + startup and transfer costs + expected recovery cost

We built deterministic replay to compare policies against the same job arrivals and realised prices. Each policy maintains independent state, and forecasts can only use information available at decision time. We also retained a persistence-forecast optimiser internally to isolate the contribution of ML.

For execution, we used the Docker SDK with predefined tasks, allowing the application to launch and monitor a real local CPU workload.

Challenges we faced

The hardest challenge was discovering that better predictions do not automatically produce better scheduling decisions.

Our initial optimiser sometimes waited for tiny predicted price reductions. Those delays consumed deadline slack and created congestion, causing it to complete fewer jobs. Its lower total spend could therefore look like a saving even when it had simply done less work.

We addressed this in two ways. First, we made comparisons account for completed workloads, missed deadlines, failures and unfinished jobs rather than just raw expenditure. Second, we introduced a cautious waiting policy that discourages speculative delays when the potential benefit is small or compatible jobs are already queuing.

Historical replay introduced another challenge: preventing future information from leaking into decisions. We added model warm-up history, source and artifact validation, reproducible inputs, and independent accounting audits.

What we learned

We learned that an ML system needs to be evaluated against the decision it supports, not only its prediction error.

Across our broader experiments, cautious ML scheduling often matched simpler policies rather than beating them. The selected compute pools offered limited price variation, and longer deadlines alone did not create much additional opportunity.

A retrospectively selected historical example demonstrates the mechanism: genuine ML forecasting helped complete the same 14 simulated jobs for $4.7819, compared with $4.8115 for the baselines. We present this as a selected success example, not evidence of consistent outperformance.

That distinction became an important part of the project: building an autonomous system also means building the evidence needed to understand when its decisions help.

What’s next

We want to evaluate Arbitr with real workload traces and more verified, interchangeable compute pools, where both timing and routing offer meaningful choices. Our AWS pricing integration provides a foundation for this; provisioning and executing workloads on paid cloud infrastructure remain future validation steps.

We also plan to add multi-provider compatibility, allowing Arbitr to compare compatible compute options across cloud providers. These comparisons would account for hardware, regional availability, transfer costs and execution requirements alongside price.

Finally, we want to support finer scheduling intervals and early resource release while retaining hourly price forecasts. Our goal is to turn the prototype into a system that can demonstrate reliable savings while keeping deadlines, execution safety and transparent evaluation at its core.