A computer’s newfound ‘intuition’ beats world poker champs

Poker expert Martin Sturc may have finally met his match: a computer program called DeepStack.

“You always have to be one step ahead,” said Sturc, president of the Austrian Pokersport Association. “When I realized he was adapting his gameplan, I had to adapt.”

Using a type of artificial intelligence its creators describe as “intuition,” DeepStack widely defeated 10 out of 11 professional players from the International Federation of Poker in a game of heads-up no-limit Texas Hold ‘em. Each player played 3,000 hands against DeepStack.

Sturc was the 11th player. DeepStack won by a margin that was too small to decide the overall winner; statistically, Sturc may be the one player who is still better than DeepStack.

“It’s a huge compliment,” he said. “It feels good that there’s a small chance that I’m still at least on the same level as artificial intelligence.”

The team of computer scientists that invented DeepStack say the program mirrors real-life decision-making more than traditional AI, and it may have uses beyond the poker table.

A high-tech digital world

To understand why DeepStack is so special, you have to understand the nature of the game, said Michael Bowling, one of DeepStack’s creators and the corresponding author on a paper published today in the journal Science.

Bowling said that in most AI games, such as the chess-playing Deep Blue and Google’s AlphaGo, the board has all the information you need to make a decision.

“There isn’t some knowledge in one of the other players’ heads that you wish you knew,” said Bowling, a computer science professor at the University of Alberta.

But in Texas Hold ‘em, the players can’t see each other’s cards. They all have a different view of the game. According to Bowling, this more closely mirrors the decisions we make in everyday life, which are often based on incomplete information.

“It’s actually a rare moment when we say, ‘Oh, yeah, I have all the information I need to make this decision,’ ” Bowling said.

Online poker sites have been around for years, and poker-playing AI is not necessarily new. But heads-up no-limit Texas Hold ‘em is a different game altogether. Because there are no limits on the sizes of individual bets, the number of possible decisions a player can make is astronomically high: 1 followed by 160 zeroes.

“(That’s) more than there are atoms in the universe,” Bowling said.

A common estimation for the number of atoms in the observable universe is roughly 1 followed by 80 zeroes. When poker bets are fixed – a variation called heads-up limit – the number of zeroes drops to just 14.

With so many possible choices, this version of Texas Hold ‘em would be too complex for most computer programs.

DeepStack “is different from ‘good old-fashioned AI,’ ” said Vlad Firoiu, a doctoral candidate at MIT’s Computer Science and Artificial Intelligence Laboratory, who recently developed an AI for “Super Smash Bros. Melee” that beat top players of the popular Nintendo fighting game. Firoiu was not involved in DeepStack.

Whereas traditional AI calculated all the possible outcomes of a game beforehand, that was not possible or practical for a game with so many possible choices. So Bowling and other modern gamemakers have turned to a newer type of algorithm called deep learning. The more examples they feed a computer, the smarter it gets at handling situations it has never seen before.

“Where we get (intuition) in DeepStack is very similar to how a human might get it … through experience,” Bowling said.

Your intuition, it will lead you in the right direction

To form DeepStack’s so-called intuition, Bowling and his team ran millions of test games against the AI. Those simulations fed into a digital brain, called a “neural network,” which has been described in the detection of skin cancer and even Google’s dreamlike artwork.

DeepStack can’t play to the end of the poker game, so it looks a few plays ahead and compares the game to similar matches it has played in the past.

In cutting the game down into a fewer number of choices, Bowling and his team created an algorithm that can run on many commercially available laptops.

However, “I’d be very surprised if the algorithm … has anything more than just superficial analogy to what we would call human intuition,” said Scott Huettel, a professor of psychology and neuroscience at Duke University. He was not involved in developing DeepStack.

Intuition is a gut feeling, a way we reach decisions without conscious reasoning, Huettel said. It is difficult to study, he said, because it happens so quickly, and it’s something we can’t quite put our finger on.

“Expert poker players may be able to ‘intuit’ properties of the people they’re playing with, but … the reason they’re an expert is not because they have great intuition. It’s because they understand odds,” said Huettel, who has used poker to study decision-making in the human brain.

Of course, DeepStack can’t read the other player’s posture, notice sweat on their forehead or call their bluff. Nor does it have tells of its own.

“A lot of what can be attributed to psychology of the game actually falls out of the mathematics,” Bowling said, adding that players may bluff to hide bad cards, but those are situations DeepStack has already accounted for through math.

“They want to win at poker by preventing the opponent from gaining any advantage,” Huettel explained.

Still, DeepStack succeeded where others failed; in 2015, an AI named Claudico lost to a team of professional poker players at the same version of Texas Hold ‘em.

“It’s an incredibly impressive accomplishment, regardless of whether there’s anything like true intuition involved,” said Huettel.

Post-modern fad

Interest in new forms of AI has been growing, such as IBM Watson’s televised win against Jeopardy! champions and Microsoft’s racist chatbot, which unintentionally learned hate speech from humans on sites like Twitter.

Although games such as poker have allowed computer scientists to measure and benchmark their AI programs, Bowling and his colleagues also look forward to a future in which their algorithm can be adapted to real-world situations – like medical care.

Bowling has started developing a mathematical model to improve care in patients with diabetes, but “we’re miles away from thinking about real trials on people,” he said.

But Huettel doesn’t see poker and medical decision making as similar problems.

“Poker is a zero-sum game,” he said. “Medical decision making usually involves aligned interests.”

In medicine, he added, even if we know the best treatments, “the challenge will be getting people to follow those rules.”

MIT’s Firoiu said that although DeepStack’s algorithm may be similar to those of newer games, it also uses many “hacks” that are poker-specific.

Medicine would present additional challenges because we don’t know exactly how many diseases work, he said, as opposed to the simpler, fixed rules of poker.

But poker expert Sturc said that DeepStack still has plenty of unreached potential within poker itself. He said it has created not just a new way to train for competitions but a way to reliably compare players before a game.

Sturc, who studied poker and AI in graduate school, said that his approach to DeepStack was to think like a computer.

“I didn’t play it 100% like I would a human being,” he said. “DeepStack does a couple of things that human players don’t do.”

DeepStack would bet more than Sturc expected, making it difficult to tell whether it had a good hand or was simply “bluffing.” Sturc described its style as “unconventional … but very solid.”

The next time they meet, however, DeepStack might be improved and harder to beat than ever.

One thing DeepStack does not do, Bowling said, is actively learn from an opponent’s strategy during gameplay. But it’s something his team is looking into, said Bowling – who, despite being an expert in AI and machine learning, is not much of a poker player himself.

“I’m a terrible poker player,” he said, “so I need a computer to play it for me.”