Blueprint: Adaptive Short-Deck PLO5 AI

Blueprint is an autonomous AI agent for Heads-Up Short Deck 5-Card Pot-Limit Omaha. The game uses a 36-card deck, gives each player five private cards, and requires players to use exactly two hole cards and three community cards.

Our agent combines an offline-trained near-equilibrium strategy with online opponent modeling. It starts with a robust blueprint strategy, observes an opponent’s betting behavior, and selectively switches to a specialized response strategy when there is enough evidence that the opponent follows a predictable pattern.

Inspiration

Poker is an especially interesting environment for artificial intelligence because the agent must make decisions with incomplete information. It cannot see the opponent’s cards, must reason about uncertain future boards, and has to balance aggression, risk, and deception.

We wanted to build an agent that combined two traditionally different approaches:

  1. A stable strategy that is difficult to exploit.
  2. An adaptive strategy that can take advantage of an opponent’s weaknesses.

A purely equilibrium-based bot may fail to exploit predictable behavior. A purely exploitative bot may overreact to randomness and become easy to exploit itself. Blueprint was designed to combine the strengths of both approaches.

What Blueprint Does

Blueprint supports all tournament actions:

  • FOLD
  • CHECK_CALL
  • RAISE_HALF
  • RAISE_POT

For every decision, it:

  1. Reconstructs the current betting state from the action history.
  2. Evaluates the player’s five-card Omaha hand under the exact 2-hole-card and 3-board-card rule.
  3. Converts the situation into a compact strategy abstraction.
  4. Looks up the appropriate action policy.
  5. Updates its model of the opponent after completed hands.
  6. Switches to an exploitative response only when the evidence is strong enough.

If the opponent model becomes unreliable or the exploitative strategy performs poorly, Blueprint returns to its baseline strategy.

How We Built It

The core strategy was trained offline using external-sampling Monte Carlo Counterfactual Regret Minimization (MCCFR).

MCCFR is well suited to imperfect-information games because it learns strategies by repeatedly sampling possible game trajectories and updating counterfactual regrets. Instead of storing every possible card and betting sequence explicitly, we created a compact abstraction of the game state using:

  • Betting history and action sequence.
  • Current street.
  • Pot and betting context.
  • Position.
  • Board texture.
  • Hand-strength buckets.
  • Previous raising behavior.

The training pipeline produces strategy tables that are exported as compact NumPy files. During gameplay, the agent does not run expensive MCCFR iterations. It performs fast feature computation and table lookup, allowing it to make decisions within the tournament’s strict CPU and time limits.

The runtime server is implemented with FastAPI and Uvicorn. The tournament sends observations to the server, and the agent returns one legal poker action. Strategy tables are loaded during startup so that the first decision does not incur an expensive initialization delay.

Opponent Modeling

Blueprint also maintains likelihood scores for several predefined opponent profiles. After observing an opponent’s actions, it evaluates how well each profile explains their behavior.

When enough actions have been observed, Blueprint compares the best-fitting profile against the baseline model. If the evidence is sufficiently strong, it activates a restricted Nash response trained against that opponent archetype.

This strategy switch is deliberately conservative. The agent requires enough observations, demands a meaningful improvement in model fit, and uses a stop-loss rule to abandon an exploitative strategy if it begins losing too much.

Technical Architecture

Tournament Judge
       |
       v
FastAPI Agent Server
       |
       +--> Observation Parser
       |
       +--> Betting-State Replay
       |
       +--> Omaha Hand-Strength Abstraction
       |
       +--> Blueprint Strategy Table
       |
       +--> Opponent Profile Likelihoods
       |
       +--> Specialized Response Table
       |
       v
Legal Poker Action

The repository includes:

  • The tournament-compatible agent.py server.
  • Precomputed strategy and policy tables.
  • The MCCFR training pipeline.
  • Baseline opponent bots.
  • A local match judge.
  • Docker build and smoke tests.
  • Runtime parity and strategy validation tests.

Challenges

Making imperfect-information reasoning fast enough

The tournament provides only a few milliseconds per decision on average and limits each bot to approximately one CPU thread. Full online search was not practical.

We addressed this by moving expensive computation offline, using state abstractions, precomputing strategy tables, loading all data at startup, and dynamically reducing board-completion sampling when the time budget becomes tight.

Preserving Omaha’s exact hand rules

In 5-card PLO, a player must use exactly two hole cards and exactly three board cards. A hand evaluator that accidentally allows arbitrary combinations can produce strategically incorrect decisions.

Blueprint explicitly evaluates the legal Omaha combinations and preserves the special Short Deck rankings, including the fact that a flush beats a full house and three of a kind beats a straight.

Avoiding premature exploitation

Opponent modeling is vulnerable to small-sample noise. A few aggressive or passive actions do not necessarily reveal a real strategy.

To reduce overfitting, Blueprint waits for sufficient evidence, compares the candidate profile against the baseline, and only activates a response when the improvement is statistically meaningful. It also monitors performance and returns to the blueprint when exploitation is no longer effective.

Keeping training and runtime behavior consistent

The training engine and the runtime agent must interpret cards, board texture, betting sequences, and hand buckets identically. Even a small mismatch could cause the deployed bot to use the wrong policy.

We added parity tests that compare runtime feature construction with the training representation, along with local simulations and Docker smoke tests that reproduce the tournament environment.

Accomplishments

  • Built a complete tournament-compatible poker agent.
  • Implemented an offline external-sampling MCCFR training pipeline.
  • Created a near-equilibrium blueprint strategy.
  • Added adaptive opponent profiling.
  • Trained specialized response policies for opponent archetypes.
  • Implemented a conservative strategy-switching mechanism.
  • Added time-budget-aware runtime computation.
  • Built a local judge for duplicate-deal match evaluation.
  • Added Docker build checks, smoke tests, and runtime parity tests.
  • Kept inference lightweight using precomputed NumPy strategy tables.

What We Learned

The biggest lesson was that strong decision-making in imperfect-information games requires both robustness and adaptation.

A strategy that only tries to exploit the opponent can overfit to short-term randomness. A strategy that never adapts can leave value on the table against predictable opponents. The most reliable approach was to begin with a strong baseline and treat exploitation as a hypothesis that must be supported by evidence.

We also learned that abstraction design is as important as the learning algorithm itself. MCCFR becomes much more practical when the game is represented using meaningful features such as hand strength, board texture, pot context, and betting history.

Finally, we learned that deployment constraints fundamentally influence algorithm design. A theoretically stronger approach is not useful if it cannot start within the startup limit, make decisions within the time bank, or run reliably in a restricted container.

What's Next

Future improvements include:

  • Training against a larger and more diverse population of opponents.
  • Learning opponent profiles continuously instead of relying only on predefined archetypes.
  • Improving abstractions for rare Short Deck PLO situations.
  • Adding stronger uncertainty estimates before switching strategies.
  • Using population-based training against evolving agents.
  • Refining the strategy tables using tournament match results.
  • Exploring deeper response strategies while preserving the runtime budget.

Built With

  • counterfactual-regret-minimization
  • fastapi
  • mccfr
  • monte-carlo-simulation
  • numpy
  • python
  • pytorch
  • uvicorn
Share this project:

Updates

Submission history