The Relationship Between AI and Poker

What is Artificial intelligence?

Artificial Intelligence is used to help computers complete tasks usually carried out by humans. From simple programs through to computers that can learn and think for themselves, AI is a new and upcoming area of research that is being used to revolutionize the world of security, transport, medicine, and gaming to name a few.

AI in gaming

AI has traditionally been used in two-player, zero sum games (i.e. where what one player loses, the other player wins) such as Go, Chess, and Checkers. Each of the AI programs designed for these games was attempting to reach some sort of approximation to the Nash Equilibrium and are based on elements of game theory. 

Games where AI has been successful tend to be ones where there is perfect information: for example, on a chess board, both players can see all pieces and identify what steps they and their opponent can take next. There may be a surprise move from an opponent, but it must be a legal move thus limiting the element of uncertainty, and AI has used this to its advantage.

What are the unique challenges with poker?

Poker is a game of imperfect information; the other players do not know what their opponents are working with and requires an element of intuition from players to guess what cards their opponent is holding in their hand. Often AI will run algorithms to calculate all possible results of a game, and in doing so can lead to strategies that don’t work for the game emerging. One of the two programs that have finally managed to beat a human poker player is DeepStack, and one of the key elements to their success has been to only have the AI calculate a few steps ahead, and recalculate data as new information in introduced to the game. The development of ‘neural networks’ created by the machine-led learning has helped the program limit the number of potential situations by learning and being able to adapt to the behavior in the game. It is estimated that training DeepStack’s neural network required the program to solve over 10 million randomly generated poker game situations.

Poker is also a game traditionally played with more than two people, and the bots have had to learn the rules of poker to be able to compete against real people.  

Artificial Intelligence Programmes in Poker

Pluribus

Pluribus was designed to play against copies of itself without any data or prior human input. It started playing from scratch and using counterfactual regret minimization as it played. This then became an iterative process and as it played it was learning and storing information on different strategies that could be used. By assessing what strategies worked, and which didn’t it was able to improve the longer it played, improving the outcomes exponentially. When pitted against real poker players, Pluribus used its stored data as a blueprint to start the game, and then the inbuilt learning software to learn from its opponent to improve its strategy. Pluribus was able to beat players (poker champions at that!) in a six-player game, not just one on one, in part due to the ability to learn and adjust and use different strategies throughout the games.

DeepStack

Deepstack was recently tested against 33 professional poker players over the course of 4 weeks, playing almost 45 thousand games of no-limit, two player, Texas Hold’em. After removing instances of winning through pure luck, researchers were able to compute that the AI had a final win rate of 486 mili-big-blinds per game, around ten times what is considered a good rate by a competent professional poker player. One of the ways it managed to do this was by having technology built in that allowed for the probabilities and strategies to be recalculated throughout the game whenever there was the introduction of new information, be it a new card of a new bet. DeepStack also has deep learning, software to help it learn as it plays.

Libratus

Designed by researchers at Carnegie Mellon, Libratus took part in a 20-day poker competition where it was able to beat four top ranking human poker players over the course of 120,000 hands. Unlike DeepStack, Libratus requires a lot more computing power for its algorithms as it does not have any kind of learning software, so it does not improve over time.

Could AI change poker?

Whilst there have definitely been huge strides in AI as technology has improved, there is a still a long way to go for these bots to take over poker games, or even really influence the game as it stands. Even in chess, where AI has been under development for years, it is only recently that programs have been designed that can beat humans, and even then, the human element of game play is still hugely important for competitors. In terms of things like bluffing, and tells, these remain hugely important in poker, and someone is a good player not just because of their hands, but due to the way they play, when they fold, the size of their bets and showing human emotion. Much of poker is rooted in psychology and socialization, aspects of the game that the bots have yet to master. AI however, can be used to help players learn different strategies and improve their poker games without risking real money.