Budget Workflow Orchestrator
History of past budget workflows
Architecture

Predictive Rate-limiting Orchestrator

A token budget governance layer that re-forecasts an agent workflow's token cost after every task and stops for human approval before the run is projected to overspend

1. Problem Statement

Typical agentic platforms such as Codex and Claude Code provide basic token rate-limiting solutions like Request-Per-Minute (RPM) or Token-Per-Minute (TPM), which while may be sufficient in limiting excessive usage and cost, only restrict usage after the damage has been done.

So, if an agent run is on track to consume five times its budget, typical rate-limiting system will only trigger when actual tokens are spent.

Even if agents were asked to estimate its token usage before executing its plan, these estimate would be wildly inaccurate. A study of token consumption in agentic coding tasks found that frontier models predict their own token usage only weakly (Pearson correlations up to about 0.39) and systematically underestimate the real cost (Bai et al., 2026).

Therefore, simply asking the agent to estimate its own token usage is unrepresentitive of the actual work that needs to be done.

2. Proposed Solution

Our proposed solution is a predictive rate-limiting orchestrator middleware that ingests planned tasks with work estimates (similar to a planner-orchestrator system), and re-forecasts the projected workflow's total token usage after every task. If the projected token usage is above the token budget, the workflow will pause and require user confirmation before continuing.

Unlike asking the planning agent to predict an absolute token cost, planning agents in this system are required to state, in relative terms, how much of the total effort each sub-task should represent. This allows for more flexibility and allows token usage to fluctuate above or below forecasts as long as it stays within the budget.

Therefore, this system effectively projects and rate-limits agent executions based on current token usage. Ensuring that usage constraints are enforced before any real damage is done.

Architecture

3. Limitations

For this hackathon, the primary aim was to build the middleware that provides token projections, dispatches agents to complete sub-tasks, and rate-limits agent execution. As a result, several non-essential services were either ignored or simplified.

The Planner agent has only a single planning turn, so it does not support an interactive back-and-forth behaviour to clarify requirements.
The solution built for this hackathon focuses solely on the middleware that enforces work estimates. The Planner agent's work-estimate accuracy was a separate problem we deliberately left aside, as tackling both the enforcement middleware and estimate accuracy would have ballooned the scope well beyond the hackathon.
The orchestrator is built to support a single-agent sequential workflow. However, the same concept can be integrated into a multi-agent system.

4. Example Use case

An operator wants an agent to add a health endpoint to a small service. They decompose it and assign relative weights (integers, 1–10):

#	Task	Weight	Meaning
1	Inspect the workspace and summarise the layout	1	quick, cheap
2	Implement the endpoint	4	the bulk of the work
3	Add a test and run it	3	substantial
4	Print the final file and report test status	2	wrap-up

Total weight 10. They set a token budget of 12,000, which could be their company's token usage policy.

Task 1 runs. It was supposed to be a 10th of the effort, but the workspace is larger than expected and the agent reads a lot of it: the reported usage is 4,200 tokens. The middleware now has a rate: 4,200 tokens for 1 unit of weight. Projected remaining cost is 4,200 * 9 = 37,800; projected total is 42,000 against a budget of 12,000.

Before task 2 is sent to the runner, evaluateBudget returns PAUSE. The workflow status becomes PAUSED_BUDGET_APPROVAL, a PAUSED event is written with the numbers that justify it, and the agent is released. The operator sees: consumed 4,200, projected total 42,000, budget 12,000, and the reason string. They can either raise the budget and resume, or force-resume without raising the budget.

Therefore without this middleware, tasks 2–4 would have run and the operator would have found out at 40,000-plus tokens that the estimate was wrong.

5. Learning Points

Knowing how difficult the previous year's TikTok TechJam problem statements were, the team joined this hackathon with zero expectations of actually being able to publish anything. Honestly, 4 out of the 5 problem statements are completely out of our skill range.

But seeing the "Agent Launchpad" problem statement, we reflected on how we were using coding agents for school and work, and identified many pain points. Some ideas were too trivial, too complex for the timeframe, or we simply did not know where to start. But after some brainstorming, we came up with solving this problem of rate-limiting.

Settling on a problem to solve, we explored how existing planner-orchestrator systems work and how our rate-limiting systems can be integrated into them. Having a short timeframe to build everything, we decided to simplify any non-essential services and only focus on the core functionality, the predictive rate-limiting orchestrator.

Through this hackathon, we have learnt a lot more about AI systems and the architecture behind them.