News for hackers. Stuff that matters.
Re:Professional Poker Players Know the Optimal Strategy but ... by mxwsn on Friday July 19, 2024 @12:20AM (#41001981)
Essentially all AI work I've seen in games aims for game theory optimal play, but I think it could be really interesting to consider AI for exploitative play. Does this exist? Poker with imperfect information, human pressure and fallibility means that players will inevitably stray from Nash equilibrium. The decision on how to exploit without getting exploited back oneself seems really fascinating to consider from an AI perspective. At a glance it seems to require considering how others view you..
Re:Professional Poker Players Know the Optimal Strategy but ... by waprin on Friday July 19, 2024 @01:09AM (#41002139)
Within solvers, you can do something called "node locking", which means you "lock" a tree in the game node to play a fixed strategy. You would typically lock it to play as you suspect your opponent plays. This lets the solver calculate the optimal exploitative solution against your specific oppoents.

Piosolver, the first public solver and the one mentioned in the article, has this feature.

However, what often happens is if you lock one node, then several other nodes in the game tree over-adjust in drastic ways, forcing you to lock all of the, which may be infeasiable. As a result, Piosolver recently introduced "incentives", which gives a player in the game an additional incentive to take a certain action . For example, you may suspect your opponent calls too much and doesn't raise enough, so you can just set that incentive and it will include that in its math equations and give you something similar to an exploitative solution with a much simpler UX.

This feature was literally just introduced a few months ago so it's still very much an active area of research, both for game theory nerds, and people trying to use the game theory nerd research to make money !

Re:Professional Poker Players Know the Optimal Strategy but ... by Buttons840 on Friday July 19, 2024 @08:47AM (#41005945)
I want to see strong AI used in video games, especially strategy games. People often retort that strong AI is not fun; it's too challenging and that's not what players want. But once we have a strong AI we can adjust its goal function in fun ways. What you're describing is effectively the same, and it's the first time I've seen this used in a strong AI.
Re:Professional Poker Players Know the Optimal Strategy but ... by currymj on Friday July 19, 2024 @02:38PM (#41009665)
GT Sophy is now a permanent feature in Gran Turismo and basically does this!
Re:Professional Poker Players Know the Optimal Strategy but ... by Buttons840 on Saturday July 20, 2024 @12:47PM (#41017775)
Thanks for bringing that to my attention.

I still haven't seen an AI for a turn based strategy game. There's AlphaStar, but it wins via APM, not strategy.

Re:Professional Poker Players Know the Optimal Strategy but ... by dontlikeyoueith on Friday July 19, 2024 @03:39PM (#41010277)
In essence, you need strong (probably unbeatable) game AI in order to make more interesting weak (beatable but challenging and fun) game AI.
Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Friday July 19, 2024 @02:33AM (#41002528)
Yes, this exists! Look up models based on counterfactual regret minimisation - they learn to exploit regularities in their opponents play, and often stray from the GTO play when it makes sense. I believe they have beaten poker professionals in thousands-of-hands playoffs but I may be misremembering.
Re:Professional Poker Players Know the Optimal Strategy but ... by phreeza on Friday July 19, 2024 @06:16AM (#41004474)
The obvious follow up question: are there methods in use to bait such models into suboptimal play and then switch play style to exploit that?
Re:Professional Poker Players Know the Optimal Strategy but ... by codethief on Friday July 19, 2024 @08:34AM (#41005807)
> I believe they have beaten poker professionals in thousands-of-hands playoffs

I really don't know anything about poker AIs but could it be you are referring to Libratus and/or Pluribus[0]?

[0]: https://noambrown.github.io/

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Friday July 19, 2024 @09:08AM (#41006136)
To be honest I didn't have a specific AI in mind, more the technique, but it sounds like these ones also use a variant of CFRM.
Re:Professional Poker Players Know the Optimal Strategy but ... by energy123 on Friday July 19, 2024 @01:05PM (#41008587)
I do not believe these attempt to exploit regularities.
Re:Professional Poker Players Know the Optimal Strategy but ... by currymj on Friday July 19, 2024 @01:08PM (#41008618)
no, CFR is mainly just a way of computing Nash equilibria and (although in some sense it is an online, iterative algorithm) would typically be used to precompute Nash strategies, not update them in real time. real poker playing systems augment the CFR strategies with some real-time solving, but just to get even closer to Nash at the end of a hand.

on top of this, you could think about augmenting these systems to exploit weaknesses in opponent strategies. there is some work on this, but I don't think it's done much. The famous systems that played against professionals don't use it, they just try to get as close to GTO as possible and wait for opponents to screw up.

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Saturday July 20, 2024 @03:26AM (#41014756)
Hmm, I see, thanks for the reply. My mistake - I watched an interview (that I can't find now, ugh) with a poker player who played against one of the top CFRM bots and claimed that it felt like it was adapting to his playstyle.

But it sounds like that must have been either misunderstanding or some other part of the bot's algorithm I guess.

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Saturday July 20, 2024 @06:25AM (#41015426)
...in case you would be willing to share some knowledge - what exactly is a GTO play in poker? Does it mean a Nash equilibrium strategy? Something else entirely?

Whenever I search this stuff I get practical poker strategy guides, but none of them seem to define the term haha

Re:Professional Poker Players Know the Optimal Strategy but ... by kuboble on Saturday July 20, 2024 @01:48PM (#41018239)
Gto poker = whatever solvers say.

Common expression is "deviate from GTO" where you know what the solver would do but decide to play differently.

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Sunday July 21, 2024 @01:47AM (#41022715)
I see, thanks, so I guess it depends on what the solver is actually doing.
Re:Professional Poker Players Know the Optimal Strategy but ... by currymj on Sunday July 21, 2024 @06:27PM (#41028824)
Two player poker is a zero sum game, where GTO play is very well-defined as just playing a Nash equilibrium strategy. The solvers try to get as close as they can to that.

Life is a lot more complicated in multiplayer poker. There are Nash equilibria, but potentially many with different payoffs, and you can't force your opponents to choose the one you're aiming for. So in that case, it's not so obvious what "optimal" means.

As for CFR adapting to opponent play: CFR could bias its compute resources towards really finely optimizing strategies for the most likely scenarios facing certain players, and it seems like this has been done during poker tournaments.

But within those situations, it would still be trying to more perfectly approximate the Nash strategy, vs. more experimental approaches which actually choose a different strategy to exploit opponent weaknesses.

Re:Professional Poker Players Know the Optimal Strategy but ... by genewitch on Friday July 19, 2024 @03:05AM (#41002713)
it took me a while to track this down last month: https://codegolf.stackexchange.com/questions/tagged/king-of-... there's also cops and robbers and at least one other "all AI compete against eachother" with the submitter usually making the first couple of "naive" bots.
Re:Professional Poker Players Know the Optimal Strategy but ... by nhggfu on Friday July 19, 2024 @08:27PM (#41012934)
very interesting share ^ !
Re:Professional Poker Players Know the Optimal Strategy but ... by asdasdsddd on Friday July 19, 2024 @01:32AM (#41002221)
I think it'd be interesting to see if an AI with visual input playing exploitatively can out perform AI playing GTO. In doing so, we can measure the effect of visual tells.
Re:Professional Poker Players Know the Optimal Strategy but ... by tialaramex on Friday July 19, 2024 @02:05AM (#41002331)
You mean, can the exploitative strategy take money from fish faster? Yes. But it doesn't need to care about visual tells.

The point of the optimal strategy is that it's unexploitable so you can disregard the other player's actions (in the game or outside it) entirely.

All exploitative strategies are in turn exploitable.

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Friday July 19, 2024 @04:03AM (#41003198)
I think you have to be careful with saying stuff like "optimal strategies are unexploitable", because it usually means "unexploitable in a particular game theory sense".

Whether the assumptions of the Nash equilibrium (or any of the others) make sense for your situation in a game of poker is an empirical question, right? It's not a given that playing a NE means you'll be "perfect" in the human sense of the word, or that you'll get the best possible outcome.

The best superhuman poker AIs at the moment do not play equilibriums either, for instance.

Re:Professional Poker Players Know the Optimal Strategy but ... by tialaramex on Friday July 19, 2024 @09:24AM (#41006305)
I agree that because of, for example rake or table fees for cash games or competition structure for tournament in practice a game theoretically optimal choice may not be the right choice in practical play.

However the situation with an AI powered competitor which uses exploitative play is identical to a human, the GTO play will gradually take their chips at no risk.

It's not that they're optimal but that they've chosen not to be optimal and so that's why they lose money against GTO.

The AI is at least unemotional about this, humans with a "system" easily get tilted by GTO play and throw tantrums. How can it get there with KToff? What kind of idiot bluffs here with no clubs? Well the answer will usually be the one that's taking all your chips, be better. Humans used to seeing exploitable patterns in the play of other humans may mistake ordinary noise in the game for exploitable play in a GTO strategy and then get really angry when it's a mirage.

Re:Professional Poker Players Know the Optimal Strategy but ... by bubblyworld on Friday July 19, 2024 @11:50AM (#41007694)
Right, I see what you're saying, but this is what I'm disputing - in two player games, what you wrote is true, but those properties of Nash equilibria don't generalise.

When there are more players, there can be multiple Nash equilibria, and (unlike the two player case) combinations of equilibrium strategies may no longer be an equilibrium strategy. So it's no longer true that you cannot be exploited, because that depends on other player's strategies too, and you cannot control those.

(See this paper for instance: https://webdocs.cs.ualberta.ca/~games/poker/publications/AAM...)

Re:Professional Poker Players Know the Optimal Strategy but ... by tialaramex on Friday July 19, 2024 @02:32PM (#41009591)
Yes, I agree that more players makes the theory at least extremely difficult and perhaps imponderable. That paper was interesting, thanks
Re:Professional Poker Players Know the Optimal Strategy but ... by raincole on Friday July 19, 2024 @12:26AM (#41002000)
> Imperfect information, human pressure... inevitably stray from Nash equilibrium

Human pressure yes. Imperfect information no.

When we talk about Nash equilibrium for a game like poker, it's already based on imperfect information.

Re:Professional Poker Players Know the Optimal Strategy but ... by mxwsn on Friday July 19, 2024 @02:06PM (#41009289)
Ah right, thanks
Re:Professional Poker Players Know the Optimal Strategy but ... by ggjkvcxddd on Friday July 19, 2024 @06:12AM (#41004447)
An AI that plays a fixed exploitative strategy will end up getting figured out relatively quickly and counter exploited pretty hard. This actually happens in real life sometimes when people attempt to deploy poker bots online.

Any exploitative AI also needs the ability to adjust in real time to a different exploitative strategy, which also needs to be not easily predictable, etc.