Professional poker players know the optimal strategy but don't always use it

https://www.scientificamerican.com/article/the-nash-equilibrium-is-the-optimal-poker-strategy-expert-players-dont-always-use-it/

by alexahn • 2 years ago

146 85 2 years ago

scientificamerican.com

mxwsn2• 2 years ago

Essentially all AI work I've seen in games aims for game theory optimal play, but I think it could be really interesting to consider AI for exploitative play. Does this exist? Poker with imperfect information, human pressure and fallibility means that players will inevitably stray from Nash equilibrium. The decision on how to exploit without getting exploited back oneself seems really fascinating to consider from an AI perspective. At a glance it seems to require considering how others view you..

raincole• 2 years ago

Imperfect information, human pressure... inevitably stray from Nash equilibrium

Human pressure yes. Imperfect information no.

When we talk about Nash equilibrium for a game like poker, it's already based on imperfect information.

mxwsn2• 2 years ago

Ah right, thanks

waprin6• 2 years ago

Within solvers, you can do something called "node locking", which means you "lock" a tree in the game node to play a fixed strategy. You would typically lock it to play as you suspect your opponent plays. This lets the solver calculate the optimal exploitative solution against your specific oppoents.

Piosolver, the first public solver and the one mentioned in the article, has this feature.

However, what often happens is if you lock one node, then several other nodes in the game tree over-adjust in drastic ways, forcing you to lock all of the, which may be infeasiable. As a result, Piosolver recently introduced "incentives", which gives a player in the game an additional incentive to take a certain action . For example, you may suspect your opponent calls too much and doesn't raise enough, so you can just set that incentive and it will include that in its math equations and give you something similar to an exploitative solution with a much simpler UX.

This feature was literally just introduced a few months ago so it's still very much an active area of research, both for game theory nerds, and people trying to use the game theory nerd research to make money !

Buttons8402• 2 years ago

I want to see strong AI used in video games, especially strategy games. People often retort that strong AI is not fun; it's too challenging and that's not what players want. But once we have a strong AI we can adjust its goal function in fun ways. What you're describing is effectively the same, and it's the first time I've seen this used in a strong AI.

asdasdsddd• 2 years ago

I think it'd be interesting to see if an AI with visual input playing exploitatively can out perform AI playing GTO. In doing so, we can measure the effect of visual tells.

tialaramex4• 2 years ago

You mean, can the exploitative strategy take money from fish faster? Yes. But it doesn't need to care about visual tells.

The point of the optimal strategy is that it's unexploitable so you can disregard the other player's actions (in the game or outside it) entirely.

All exploitative strategies are in turn exploitable.

bubblyworld7• 2 years ago

Yes, this exists! Look up models based on counterfactual regret minimisation - they learn to exploit regularities in their opponents play, and often stray from the GTO play when it makes sense. I believe they have beaten poker professionals in thousands-of-hands playoffs but I may be misremembering.

phreeza• 2 years ago

The obvious follow up question: are there methods in use to bait such models into suboptimal play and then switch play style to exploit that?

codethief• 2 years ago

I believe they have beaten poker professionals in thousands-of-hands playoffs

I really don't know anything about poker AIs but could it be you are referring to Libratus and/or Pluribus[0]?

[0] https://noambrown.github.io/

currymj3• 2 years ago

no, CFR is mainly just a way of computing Nash equilibria and (although in some sense it is an online, iterative algorithm) would typically be used to precompute Nash strategies, not update them in real time. real poker playing systems augment the CFR strategies with some real-time solving, but just to get even closer to Nash at the end of a hand.

on top of this, you could think about augmenting these systems to exploit weaknesses in opponent strategies. there is some work on this, but I don't think it's done much. The famous systems that played against professionals don't use it, they just try to get as close to GTO as possible and wait for opponents to screw up.

genewitch• 2 years ago

it took me a while to track this down last month: https://codegolf.stackexchange.com/questions/tagged/king-of-... there's also cops and robbers and at least one other "all AI compete against eachother" with the submitter usually making the first couple of "naive" bots.

nhggfu2• 2 years ago

very interesting share ^ !

ggjkvcxddd• 2 years ago

An AI that plays a fixed exploitative strategy will end up getting figured out relatively quickly and counter exploited pretty hard. This actually happens in real life sometimes when people attempt to deploy poker bots online.

Any exploitative AI also needs the ability to adjust in real time to a different exploitative strategy, which also needs to be not easily predictable, etc.

brigadier1326• 2 years ago

"Game theory optimal" (as poker players like to call it) is not really the optimal strategy. It's the nash equilibrium strategy assuming the other players are also playing a nash equilibrium strategy. As soon as one player deviates from the nash equilibrium it's not optimal anymore :).

It's interesting how this affects play. If a player is bluffing slightly less than they should be, the adjustment is drastic, you should actually never call with hands that do not beat their value hands. If they are bluffing optimally, you are supposed to call using what is known as "minimum defense frequency". What's interesting about this is the minimum defense frequency is based on what the strongest hands you can possibly have in that situation are and the opponents possible hands do not even factor into it. It's required to prevent your opponent from bluffing with any two cards profitably.

To do the math, if you are on the river and the opponent bets 100 into 100, for this to be profitable for them they need to win 50% of the time or more. If your opponent is bluffing optimally, you need to call with 50% of the strongest hands you have in that specific situation (if you don't know what hands you have in a specific situation that's a problem) and sometimes they can be dogshit like King high.

But, very important to note, very few players actually bluff enough and if they bluff less than they should you should only ever call when your hand actually beats their range of possible value hands. (value vs bluff is kind of a difficult thing to communicate, generally it's value if you want your opponent to call)

Most players don't bluff enough as a result of most players calling too much! When they call too much you should obviously not bluff! This leads to very boring games of poker.

waprin6• 2 years ago

What's interesting about this is the minimum defense frequency is based on what the strongest hands you can possibly have in that situation are and the opponents possible hands do not even factor into it.

This is actually not quite true. MDF is purely a formula based on pot size and the bet size (pot size / pot size + bet size). The fact that it doesn't consider various ranges is why it's not really useful - it was a simplified formula used to try to understand the game before solvers existed.

There are situations where your opponent can bet any two cards profitably and you do have to fold - imagine they bet the size of the pot, but have the better hand 99% of the time, you're simply forced to let them bluff the 1% of the time they're bluffing. MDF is a pre-solver concept and not an especially useful concept in the modern game.

brigadier1326• 2 years ago

I'm pretty sure mdf applies to rivers when you are last to act. I'd be interested in being proven wrong however if you have solver output that shows it. I remember studying solver output and seeing it in action.

I know that before the river there are range advantages that make defending mdf a losing play.

Lordarminius• 2 years ago

This is why I swear by chess. Bluffing has minimal value against best play.

tucnak• 2 years ago

Not in blitz it doesn't

jurassicfoxy• 2 years ago

That being said, when Super GMs play a bad move against a lower rated GM, they quite often gets a pass. They don't capitalize, simply because they assume the move is excellent.

ajkjk• 2 years ago

Chess manages in practice to be a game of imperfect information, like poker. Obviously it is explicitly a game of perfect information on the board, but the hidden information is all psychological. For instance: "do they actually know this opening/endgame? do they see a tactic? did they take a long time because they calculated that it's a good move, or are they bluffing by making it seem like they calculated something? are they actually better than me or just acting like they are?". etc.

It's true that "bluffing has minimal value against best play", but no human is in that situation. Even super-GMs will play "bluffs" if they are behind (or playing a lower-rated player and sure they can recover later. or just for fun if the stakes are low).

And that's not even mentioning optimal strategy under time pressure. For instance some of the Lichess tournaments are structured such that winning fast is more valuable than winning slowly because the resulting score comes from how many wins you can get in (or in other cases, how big of a streak you can get). So people will play in a way that optimizes for winning quickly by taking big bets / bluffing / creating chaos with un-calculated gambits, especially if they have a good reason to believe they're better than their opponents.

hibikir• 2 years ago

But every serious chess game is a matter of time allocation. The best players in the world are unable to calculate as much as they want in every position: Ultimately there is bluffing and risk taking. See the last game of the last candidates, where both Fabi and Ian have to win to get into a playoff, and they get themselves into extremely complicated positions, where accurate play just takes too long for a human. At an 8 hour time limit, the game is very different than at the time the players actually had, as ultimately Fabi just couldn't calculate to the end on every position he knew was key.

It wasn't the most accurate game of the tournament, but the most instructive as far as the psychology of chess goes

tialaramex4• 2 years ago

In the narrow range of poker variants (all heads up, ie only two players, not full ring like all but the last few hands of the Main Event) where it's meaningful to talk about a truly optimal game, any theoretical optimal play will still take money from all humans it's just slower (but with zero risk) compared to exploitative play.

In live cash games, speed matters, you want to take all the available chips before the fish realise they're out-matched, but to protect yourself the optimal play, if you could memorise it, would be safer because it can't be exploited yet it does take the opponent's chips.

Poker players are gamblers. So "safer" isn't really what they were going for anyway.

brigadier1326• 2 years ago

More important than speed is variance. If you really crush your opponents you are less at risk of busting from bad bankroll management.

jason-johnson• 2 years ago

”Game theory optimal" (as poker players like to call it) is not really the optimal strategy. It's the nash equilibrium strategy assuming the other players are also playing a nash equilibrium strategy. As soon as one player deviates from the nash equilibrium it's not optimal anymore :).

This is not really what most people mean when they say “optimal strategy”. It’s true that exploitative play will make more money if you know exactly what your opponent is doing and if they keep doing it despite it not working. Neither of these will generally hold in an actual game.

The reason it’s called “optimal strategy” is because it works no matter what your opponent is doing. It will not make as much as a strategy tailored perfectly to your opponent but it will never lose to anyone under any circumstances, assuming infinite games (assuming infinite games just so we can ignore variance). The worse case the strategy has is break even to anyone else using it.

pfortuny• 2 years ago

You are assuming there are no collusions among players (agreements to play against another one). That perturbs the Nash optimality.

drdeca2• 2 years ago

It sounds like you are maybe describing a dominant strategy? Oh, wait, no, you are saying that if you play the Nash equilibrium, then no other strategy among opponents will do better against it than the Nash equilibrium would, and therefore (as the game is zero sum) the worst their choice of strategy can make you get on average, is breaking even?

Ok, now that I (I think) understand your comment: I don't think you have to know exactly what non-Nash strategy someone is playing in order to exploit it. I don't think trying to estimate how someone is likely deviating from the Nash equilibrium, in order to try to exploit it, is necessarily a mistake. I think it could be feasible for someone to get better returns on average by noticing off-Nash patterns of play in other players, than playing Nash regardless would? (Not that I could win this way. I couldn't.)

brigadier1326• 2 years ago

Neither of these will generally hold in an actual game

These absolutely hold, 90% of casual poker players have the exact same strategy problems of not bluffing enough and calling too much.

Moon1024• 2 years ago

You're writing something that is at best accidentally misleading and at worst confidentially incorrect.

What is your formal definition of "optimal strategy"? A Nash equilibrium is considered optimal in the sense that it's a state where no one can gain an advantage by deviating from the equilibrium.

Sure, if your opponents don't play the Nash equilibrium, there is room to exploit that deviation and potentially gain more than what you would get from playing the Nash equilibrium. However, you also make yourself exploitable in return, so I don't think you're presenting the whole picture here.

brigadier1326• 2 years ago

Poker players typically optimize for making the most money in expectation per hand. Either way, I'm certain that the exploits I've described for players that bluff too much or call too much are correct so I'm not too worried about being slightly off. Poker strategy is about heuristics.

PeterisP• 2 years ago

For practical poker, I'd formally define "optimal strategy" as the strategy that maximizes profit per time (or per game) for a set of opponents, including also the actions needed to "explore" and discover any bias before exploiting it.

Assuming at least one of opponents is not playing Nash equilibrium (which is a very solid assumption), playing the Nash equilibrium becomes suboptimal as it doesn't exploit the exploitable as much.

14• 2 years ago

Trying to understand what you are getting at made me realize why I do not gamble I am just not smart enough or lucky enough. I had a friend try and convince me he knew a sure fire way to beat roulette but in the end the house always wins. He eventually had to admit he had a gambling addiction and quit doing it all together.

namanyayg• 2 years ago

Like anything else, Poker is a skill that you can learn with time and practice.

It is not really related to your smartness or luck (doesn't apply to _everyone_ of course but I'd wager that the average HN reader is already smart enough for poker)

tmiku• 2 years ago

Of all the gambling games to insist you know a secret solution too, roulette has to be one of the funniest.

mkatx• 2 years ago

I love poker, and I'm solid on the math foundations, but I still suck..

Any book/other resource recommendations for brushing up on this stuff?

Raidion2• 2 years ago

Modern Poker Theory by Acevedo was the premier book on this, but I've been out of the game a few years. Idk if I'd fully trust his charts given modern sizing theory, but it's going to improve your game if you understand the concepts. If you're really serious, you want to get a solver: GTO+, PIOSolver, or GTOWizard (online version).

dmurray• 2 years ago

When they call too much you should obviously not bluff! This leads to very boring games of poker.

I don't know, poker theory is all about optimal ranges and Nash equilibrium, but there's something satisfying (and very practically important, since if all your opponents even understand the phrase Nash equilibrium you should find a different game) about trying to make the most money against an opponent who calls or bluffs way too much.

gweinberg• 2 years ago

Boring but profitable. It's super easy to take money from a "calling machine": just bet big when you have a strong hand.

waprin6• 2 years ago

I'm working on a project aiming to help pro (or serious amateur) poker players learn game theory, mostly via flashcards with spaced-repetition.

https://www.livepokertheory.com

I do personally dislike that GTO became the nomenclature , as I prefer "theory-based", since it causes this confusion, but trying to fight it at this point is hopeless because GTO is the search term people are using. And when people say they "play GTO" they usually mean "equilibrium" rather than "optimal against my specific opponents" which is "exploitative".

If you actually watch what the top players advocate for, everyone suggests you want to play exploitatively. However, there's one equilibrium solution and effectively infinite exploitative solutions, so equilibrium is a reasonble starting point to develop a baseline understanding of the mechanics of the game. It's tough to know how much "too much" bluffing is unless you know a baseline.

Furthermore, if you "exploit" people by definition you are opening yourself up to being exploited so you need to be very careful your assumptions are true.

Also, with solvers like piosolver, you can "node lock" (tell a node in the game tree to play like your opponent, rather than an equilibrium way plays), but there's many pitfalls, such as the solver adjusting in very unnatural ways on other nodes to adjust, and it being impractical to "lock" a strategy every node in the tree. There's new ideas called "incentives" which gives the solver an "incentive" to play more like a human would (e.g. calling too much) but these are new ideas still being actively explored.

Rock paper scissors is frequently used to explain GTO but it's not the best example because equilibrium in rock paper scissors will break even against all opponents, but equilibrium poker strategy will actually beat most human poker players, albeit not as much as a maximally exploitative one.

There's two other huge pieces this article glosses over:

1) It's as impossible for a human to play like a computer in poker as in chess - in fact far more impossible, because in poker you need to implement mixed strategies. In chess there's usually a best move, but in poker the optimal solution often involves doing something 30% of the time and something else 70% of the time. The problem is that, not only are there too many situations to memorize all the solutions, but actually implementing the correct frequencies is impossible for a human. Some players like to use "randomizers" like dice at the table, or looking at a clock, but I find that somewhat silly since it still so unlikely you are anywhere near equilbrium.

2) Reading someone's "tells" live is still a thing. While solvers have led to online poker to decline due to widespread "real time assistance", live poker is booming (the 2024 World Series of Poker Main Event just broke the record yet again) , and in person in live poker, people still give off various information about their hand via body language. From the 70s to the early 2000s, people were somewhat obsessed with "tells" as a way to win at poker. Since computers have advanced so much, it's fallen out of favor, but the truth is, both are useful. It's totally mistaken to think that advancement in poker AI , GTO , and solvers have rendered live reads obsolete. In fact, in 2023, Tom Dwan won the biggest pot in televised poker history (3.1 million) and credited a live read to his decision, in a spot where the solver would randomize between a call and a fold.

isatty• 2 years ago

Very nice! WC3/Dota inspired streaks on the demo flash cards?

waprin6• 2 years ago

Yes indeed glad you noticed! Been too addicted to that godforsaken game at points so figured borrow some of its qualities for my studying apps...in general I'm interested in gamification + studying.

mckn1ght• 2 years ago

It’s funny they mentioned Magic: the Gathering, because the first thing I thought of when reading the headline is all the conversation I see in r/EDH, and less frequently in other related subreddits, about a reluctance or even disdain for “optimal” play, which would be trying to win as efficiently as possible. To preserve the excitement and surprise of gameplay, people discourage building decks out of “staple” cards that would quickly homogenize play. “Broken” or “busted” cards that turn out to be more powerful than anticipated by the game’s designers wind up banned in sanctioned tournament play.

ip26• 2 years ago

If you’re going to play MTG as a simple game theory optimal affair, might as well switch to poker… a Nash equilibrium surely exists all the same, but charm of the game is in the variety and checks & balances.

rcxdude2• 2 years ago

Well, from the perspective of optimal play the more complex and changing nature of MTG means that optimal play is harder to find and doesn't stay still, so there's a lot more metagame of figuring out what that play is.

rcxdude2• 2 years ago

EDH is especially prone to it as a format, because it's a format which allows almost every card ever printed (with a pretty... inconsistent banlist). This makes for some pretty wild variations in power levels, and the top end tends to be both very samey and very expensive. It's very popular as a format but it only really works because there's a culture of trying to match power levels between decks (to be fair, there's also the fact that 100 card singleton does tend to mean it's hard to be completely consistent, so there's a lot more 'bleedover' between power levels in terms of chance to win in a given match, as well as the >2 player aspect levelling things out as a player obviously in the lead will tend to be ganged up on)

willcipriano• 2 years ago

EDH is the variant of the game for people who aren't that competitive. They spend a lot of time trying not to win too hard. The format is fine but I don't like it beacuse of this culture around it, it basically has this social metagame where you can get better players removed out of play by complaining about them.

Draft, Standard and Modern are for people who want a real game without having to worry about playing too well.

thom• 2 years ago

And yet overpowered Commander cards continue to mess up Cube and eternal formats.

r34• 2 years ago

There is a huge difference between game theoretic optimal strategy and actual profitable strategy due to the human nature of the players. I imagine a professional poker player as someone who certainly knows the odds (math of the poker game in general), but is also very good in interpreting his opponents behavior (which would minimize the information revealed by them, but could they do it completely?). There are so many biases which even professional players have to overcome, that in my opinion poker is strongly psychological game. Also because of that I think that online poker and live poker are slightly different games.

thih9• 2 years ago

As a person who enjoys poker recreationally, whenever I visit these submissions I soon realize that this is a very different game from the one I’m playing.

PaulRobinson• 2 years ago

I'm a fan of the levels of Poker thinking: https://www.blackrain79.com/2020/01/outsmart-your-opponents-...

Basically, play one level - exactly one level - beyond where you peg your opponents at.

Poker is not about playing cards. It's about playing people. Cards is just how we keep it civil and not too personal.

midiguy• 2 years ago

It used to be about playing people back when no one knew anything. Now that people study GTO, it's definitely more about the cards.

kruhft• 2 years ago

Is that not called 'bluffing' with strategic planning?

kkwteh• 2 years ago

Watching Doug Polk discuss hands on YouTube really opened my eyes as to how professional pokers think about hands.

leesec• 2 years ago

Optimal in this case means least exploitable it doesn't mean most profitable, which is the real point of poker

krukah• 2 years ago

One thing I haven't seen anyone mention yet is that Nash equilibria do not actually exist when you move beyond heads-up into multi-way play. There's strong empirical evidence that solving abstracted games using MCCFR and using real-time depth-limited tree search dominates humans and outperforms all other AI strategies, but these results aren't theoretically sound, unlike in heads-up play.

I've actually been working slowly on https://github.com/krukah/robopoker, an open-source Rust implementation of Pluribus, the SOTA poker AI. What I've found interesting is the difference in how I approach actually playing poker versus how I approach building a solver. Playing the game naturally consists of reasoning about narratives and incorporating information like hand history, play style, live tells. Whereas solving the game is about evaluating tradeoffs between the guarantees of imperfect-information game theory and the constraints of Texas Hold'em, finding a balance between abstract and concrete reasoning.

waprin6• 2 years ago

Looks cool I’ll keep an eye on it.

mcswell• 2 years ago

Sort of related: I was at a restaurant last night where a sports channel was playing a poker tournament. Paint drying would have been more interesting; it was hard to tell if one of the players was even alive.

cheeze• 2 years ago

This can be said of so many things. Like a cycling race, or NASCAR, or golf, or chess.

I love it. Love cycling as well.

Ylpertnodi• 2 years ago

In each you mentioned, you can see the participants are at least awake. Love cycling too (and have no problems with the first 280km :--)

xyst• 2 years ago

I vaguely recall a poker tournament where a player employed this exact strategy and it worked out for him. I think it was the circumstances and just a bit of luck that allowed him to advance to later stages.

ziofill• 2 years ago

This is analogous to what top chess players do. They try to take the opponent into suboptimal positions hoping that they are not familiar with the best lines, even though technically the position isn’t as good as the main line.

bfeist• 2 years ago

https://archive.is/2024.07.20-035804/https://www.scientifica...

Professional poker players know the optimal strategy but don't always use it

Related Stories