Heads Up Poker Bot - team_montecarlo

Inspiration

Poker is a game of incomplete information, luck, and reading your opponent, which is what makes it such a good test for AI. When we saw that the TAMU Datathon challenge was heads-up Short Deck 5-card Pot-Limit Omaha, we were hooked. It's a variant almost nobody has solved or written much about: a 36-card deck, five hole cards, flushes beating full houses, and only 40 big blinds per stack. There were no ready-made strategy charts to copy, so we had to build our understanding from first principles and from data.

The constraints made it even more interesting. Each bot gets 8 seconds of thinking time per 1,000 hands, on a single CPU, and five errors forfeit the whole match. We had to be smart and fast and bulletproof.

What it does

MonteCarloAgent is a poker server that plays every decision by estimating the chip value of each of its four legal actions (fold, check/call, half-pot raise, pot raise) and choosing the best one.

Equity by simulation. It deals out thousands of possible opponent hands and board runouts, then measures how often it wins.
Range reading. It doesn't compare itself against a random hand. It weights the simulated opponent hands toward what the opponent's betting actually suggests, and calibrates that with the hands it sees at showdown.
Opponent modelling. It tracks how often each opponent raises, folds to raises, and re-raises, split by situation, and adapts during the match.
Strict time management. It budgets its own thinking time so it never runs out of the bank.

How we built it

1. A fast evaluator. The provided omaha_score checks all $ \binom{5}{2}\binom{5}{3} = 100 $ ways to make a hand, one dictionary lookup at a time. We precomputed a table of the score of every hole-card pair $ p $ combined with every board triple $ t $:

$$ PT[p, t], \quad p = 1, \dots, \binom{36}{2} = 630, \quad t = 1, \dots, \binom{36}{3} = 7140 $$

A hand's score then becomes a single vectorised max over a $ 10 \times 10 $ gather in numpy. That scores 512 hands in about 0.19 ms, and it matched the official evaluator exactly on thousands of random hands.

2. A preflop table. We grouped all $ \binom{36}{5} $ = 376,992 starting hands into 19,998 suit-isomorphic classes and precomputed each class's equity offline. Preflop strength is then a lookup that costs microseconds.

3. An EV policy. For a call, with equity $ E $, pot $ P $, cost $ C $, and a realisation factor $ R $ for position:

$$ \text{EV}(\text{call}) = E \cdot R \cdot (P + C) - C $$

For a raise that adds $ A $ chips from a stack of $ S $, with fold probability $ f $, re-raise probability $ q $, equity $ F $ against the hands that just call (pot becomes $ Q $) and equity $ G $ against the hands that re-raise (all-in pot $ T $):

$$ \text{EV}(\text{raise}) = f P + (1 - f) \left[ (1 - q) \left( F R' Q - A \right) + q \max \left( -A, \; G T - S \right) \right] $$

4. A testing lab. Matches are scored in duplicate: the same 1,000 deals are played twice with seats swapped, and you only win if the 95% confidence interval of your paired result is entirely above zero. With mean $ \bar{x} $, standard deviation $ s $ and $ n $ deals:

$$ \bar{x} - 1.96 \frac{s}{\sqrt{n}} > 0 $$

So we built our own tools that score exactly the way the judge does:

an in-process duplicate-match harness;
a paired A/B tester that compares two versions on identical deals, which cancels card luck;
HTML tearsheets showing where each bot wins and loses chips;
a zoo of more than 20 practice opponents: nits, maniacs, jammers, trappers, "template" bots built the way a typical AI-assisted entry probably would be, and an independently built challenger bot.

Every change had to earn its place in a statistically significant A/B test.

Challenges we ran into

The invisible fold. Our biggest bug was subtle. The bot learns about the opponent from the action history it sees on its own turn. But when an opponent folds, the hand ends immediately, so the bot never saw it. A bot that folded 70% of the time looked like it raised 99% of the time instead of its true 30%. Against an aggressive "jammer" bot, ours called almost every all-in with about 45% equity and lost around 1.4 bb per deal. Counting those hidden hand-ending actions swung that matchup to a win, and improved results across the board.
A model that over-worked. We added a look-ahead for opponent re-raises, and at first it made results worse. A soft cut-off inflated the assumed re-raise rate, and against maniacs, re-raises carry no information at all. Gating it to selective re-raisers turned it into a clear gain against bots that re-raise for value, with no loss against the rest.
Time is a resource. A tearsheet run on an overloaded machine showed what happens when the bank runs out: thousands of automatic folds. That taught us to design for an unknown, possibly slower tournament CPU, not just for our laptops.
Noise. With a per-deal standard deviation of about 25 bb, small improvements are invisible without careful paired testing. A lot of ideas that "felt" good turned out to be noise.

What we learned

Robustness beats brilliance. A bot that never crashes and never times out collects most of the available points before strategy even matters.
Measure everything. Our biggest win came from finding a counting bug in the data, not from a clever new strategy.
Opponent modelling only works if your observations are right. A sophisticated model built on biased counts is worse than a simple one.
Exploits are opponent-specific. Changes that crush one style can quietly lose to another. You only find out by testing against a varied field.

Results

Against our field of 21 opponents, the bot won 37.5 of 42 possible points. It beat every baseline bot by +6 to +12 big blinds per deal, with zero errors and only about 2 seconds of the 8-second bank used per half. It passed the official Docker build and smoke test with a 1 MB footprint on top of the base image.

What's next

Strategy switching. We built a detector that recognises when an opponent plays like our own bot, by replaying its showdown decisions through our policy. Experiments show a counter-strategy built around targeted river bluffs and stabs beats a mirror of our own bot by about +1.8 bb per deal. We want to validate it against a wider field and switch it on safely.
Fewer folds. Close the over-folding leak against aggressive bettors on the turn and river.
More independent sparring partners, so we keep finding new holes before the tournament does.