Artificial Intelligence and Man vs. Bot

“So, then because thou art lukewarm, and neither cold nor hot, I will spue thee out of my mouth.”
–(Revelation 3:16)
Not much to report, as I’ve been hangin with the old folks this week (my wife’s grandparents are visiting from Sweden). Between that and trying to get over a cold, I haven’t spent much time on the tables.
I did get to play with an excellent Hold ‘Em Simulator, thanks to the advice of a reader (thanks D). He sent me to Poki’s Poker Academy, a poker sim developed by the Artificial Intelligence group at the University of Alberta. I played on a full Poki table, and the players seemed better than the mindless bots in Turbo Texas Hold ‘Em, as one might expect when the developers incorporate cutting-edge AI technology. It also gives the option of playing heads-up or shorthanded (5 players at the table), so it’s a great tool for everybody wanting to prepare for the insane 6-max Party tables.
I probably won’t buy it– I can’t bear playing sims since I barely have any time for real poker anymore– but I wish I could have played with it rather than putting in so many hours on TTH.
A quick AI tutorial
It’s rare when I actually make use of my high-falutin Master’s degree, but I did spend several years studying AI in grad school. For any of you wondering how they “teach” these bots, it goes something like this:
1. Compile a database containing a lot of hands and their outcomes. This is called the “training set”.
2. Have a human “label” these hands based on the outcome. A simple example is labeling them as a 1 (good play) or a 0 (bad play). We might replace the human and automatically label the hands where the player won the pot with a 1 and lost the pot with a 0.
3. Once the labelling is complete, feed the hands and their outcomes to the computer. The computer will determine the variables that most affected the outcome (e.g. Number of players in the pot), and generate an algorithm based on these variables. For example, it might find a rule that says “raise with suited connectors pre-flop when there are 6 or more players in the pot”.
4. Use the generated algorithm on another database of hands, and determine the algorithm’s performance (e.g. how much did we win using this algorithm?).
5. Tweak the algorithm until we have the best possible performance.
And that’s machine learning in a nutshell. We try to find the optimal algorithm based on a mass amount of data. Obviously, computers are better at seeing patterns in huge amounts of data, and in many cases the computer’s algorithm will far exceed any human algorithm.
Of course there are lots of tweaks– the human can intervene and give the computer hints about optimal play (e.g. introduce an element of randomness to decrease predictability). But the big question is, can all of poker be broken down into discrete variables, or is the “feel for the game” something that cannot be quantified?
On one hand, a computer can instantly “play” and learn from millions and millions of hands, whereas a human must plays 140 hands a day for 20 years to reach 1 million hands (note: Grubby may get there at this rate). On the other hand, there are so many variables that may be impossible to quantify. How do we learn that a player who is cursing in the chat box after getting a bad beat is likely to be raising with nothing?
I think that poker bots could probably do pretty well at low limit poker, and perhaps could even be a strong player at higher limits. Poki has been crushing all comers in heads up matches– see the stats here and try playing Poki online. I don’t want to believe it, but heavy doses of game theory combined with mass amounts of data may be too much for even a WCP to handle. I think the WCP would probably crush Poki in the short run, but the bot’s ability to learn and adjust would be tough to overcome. Not only can the bot remember every single hand, but they can remember every single action on every single hand, instantly compile this data, and act on it. A scary opponent.
That said, my gut tells me that Poki ain’t gonna be much of a match for Doyle, Phil, TJ, or Howard. Maybe someday we’ll see it.
So how did yours truly fare? I took on “SparBot”, Poki’s successor: a state of the art heads up bot. I managed to catch some cards and win 5 BBs in the 30 practice hands, but not without some scars along the way. It beat me up early by playing supertight aggressive poker, but I figured the bot’s strategy out after a while (it folded on the turn seemingly too much) and was able to capitalize. I don’t know how I’d fare in the long run, however. After playing heads-up, I’m reconsidering paying the cash for the bot, as it’s interesting to see what the programmers came up with as the “optimal strategy” for heads up play. SparBot loved to 3 bet pre-flop, something I rarely do but probably should start doing more. I recommend everybody out their test their heads up skills against the two bots offered in the demo (only 30 demo hands, but it was fun).
Poker Blog Patrol
Chicago Phil ruminates on going pro after dominating the SNGs over the weekend. An excellent post offering some reasons not to make the leap:
“After a while, or maybe even right away, it would become a grind, just like any other job. So I would lose my favorite hobby, AND be stuck in a job with (maybe) no future, tied to my desk all day and with no real human contact (since I play mostly online). I think the prospect of that kind of lifestyle makes the whole “dream” lose its appeal.”
Newcomer Al Can’t Hang gets my vote for rookie blogger of the month, and not just because he linked me up in his latest post. Al offers a new perspective on the game, reminding overly intellectual clowns like me that the cards sometimes look better through a Southern Comfort haze:
“It was a great weekend of poker and drinking. I don’t want to calculate the volume of SoCo consumed this weekend but I can tell you that the Philip Morris Co. won’t be going out of business anytime soon.”
Old Time Poker: AlCantHang, Eddie Shore. Don’t let his self deprecating humor fool ya, the guy is a winner in poker and in drinking contests life.
Check out Bill’s tales of hole in the wall poker joints up in Northern Cali. I thought these places only existed in old Western movies.
Lord G breaks it down in detail, giving us the numbers after a tough but winning week. He’s paying his dues before he rolls up the stake and goes to Vegas. Good luck LG, I really hope this will be your big week.
The son of Halv had a monster night playing in a Big Bear cabin. I hope his luck is with him this weekend, when we take on the fish at the low limit tables in LA. We’ll both need it… the schools of fish teem and it takes every bit of poker knowledge in your head just to keep some chips on the table.
And last but not least, go sign up for the weekly poker BLOGGER tourney at Planet Poker! Ms. Felicia has put in the time to create an excellent blind structure and minimize the rake ($1!). I guess that’s good enough for me to forgive her Malmuthian quote to start off her most recent post.
How to Fight The Empire
I leave you with an email that was forwarded to me from one of my favorite bloggers. I’m sure a lot of the other bloggers have been annoyed at the constant barrage of emails from the Empire marketing army. Here’s how once blogger dealt with it:
> — Nisha wrote:
> > Hi,
> >
> > My name is Nisha, and I am working with the Marketing team of Tradal
> > LTD.
> >
> > I visited your site www.maskedblogger.com and found our Empire Poker
> > link on the same. For our mutual benefit, I would like to offer you to
> > promote Empire Poker more aggressively. We can do this by placing
> > our Link and Banner at the Top most position of your Home Page.
> >
> > We would like to welcome your suggestions on this matter.
> >
> >
> > With Best Regards,
> > Nisha Mundra
> > Marketing team
> > Tradal LTD
> > ruki@tradal.net
> Date: Tue, 23 Mar 2004 16:11:36 -0800 (PST)
> Subject: Re: PROMOTING EMPIRE POKER
> To: Nisha
>
> Dear Nisha:
>
> Thank you for the email. I would be happy to promote Empire Poker
> more aggressively, if in exchange you also put my banner and link on
> top of www.empirepoker.com. If we’re to mutually benefit, we should
> exchange banner links.
>
> Alternatively, I would accept 30 percent of the rake back on my play,
> which I feel I should be receiving anyway on a house account. If I’m
> to pursue recruiting new players for a rakeback for our mutual
> benefit, I’d at least like to get a rakeback on my own play. That
> would encourage me to play more and not switch to PartyPoker.
>
>
> Sincerely,
>
> the masked blogger