Messi v10

Inspiration

Poker requires decision-making with partial knowledge of the situation. Cards are obviously important, but so is your own position, size of the bet, and clues that your opponent leaves during hundreds of hands. We decided to create an agent that could reason about all of that within a time limit and adapt regardless of its opponent.

That's why we called our bot Messi v10 as we wanted a player who would read the game, adapt to the opposition, and do the right thing when it counts.

What it does

Messi v10 plays heads-up short-deck five-card pot-limit Omaha. At every decision, it makes up its mind whether to fold, check/call, or raise based on its own hole cards, the community cards, the pot size, history of bets, and the set of legal actions.

It considers both an estimate of its own hand strength and a model of the opponent, which is built based on his behavior throughout the current half. These are used to determine whether to put pressure on the opponent, to play carefully due to the risk of a reraise, or that the opponent calls often enough to warrant raising more hands for value.

The opponent’s hole cards are unknown to the bot. The estimate is made using simulation and public information, while adaptation is evidence-gated.

Our implementation

Messi was implemented using Python as an extension of the FastAPI and Uvicorn agent interface provided by the competition. It uses a preflop equity table to make efficient decisions on starting hands while simulating hand strength after the flop through the Monte Carlo approach using the Omaha hand evaluator provided by the competition.

There are per-half counters for keeping track of how the opponents react to betting and reraising. The bot considers this along with pot odds and equity to make decisions. When the budget for making decisions becomes tight, it uses a less costly way of estimating the equity.

We developed our own local league where duplicate hands were played from both sides of the table.

Challenges we ran into

The main obstacle was finding the right balance between exploiting our opponents' mistakes and being prudent about it. An approach which exploits one opponent will not work well against another. For instance, aggressive reraising may have been effective against players who fold too much, but it would fail against bots who play in another way.

In Five-Card Omaha, there is an enormous number of possibilities for hand and board combinations, but the agent's time to think about them is limited. Precomputed starting hands strength values, cached estimations, and a cheaper solution were employed to make the approach efficient.

Finally, having a few successful games was not sufficient for us. To make sure that our approach worked, we ran the experiment on different seeds, seats, and a variety of opponents, employing confidence intervals to separate wins from non-significant games.

Our accomplishments that we are proud of

Messi was subjected to various iterations based on the result of matchups in order to find possible weaknesses in his playing style and introduce fixes. Messi v10 has played a total of 225 games against 75 bots in a local league tournament recording, where the same 1,000 deals have been run for three seeds: 176 wins, 46 draws and 3 losses without any candidate mistakes or fallbacks recorded; all of the losses happened against Messi v2.

In a different recorded 12 seed comparison tournament against M5 Challenger, v10 had an average profit of +0.210 big blinds per deal with a reported 95% confidence interval of +0.121 to +0.299.

What we learned

The best choice is not necessarily the most aggressive one. A poker player must be able to deduce what an opponent’s action means, understand when there is insufficient evidence, and be aware of its timing through the game. In addition, the choice must work against a variety of opponents, and not just the one that it was made for.

What's Next for Messi v10

Our plan is to submit Messi v10 to the competition. Then, we would like to explore the single weakness of Messi v10 in relation to Messi v2, and work on improving our opponent range estimation.