Saturday, December 26, 2009

Robot Statistics

Most of you are BBO users, and probably know of the robot duplicate tournaments that were introduced earlier this year. You play against three robots (the program GiB, on one of its faster, less skillful settings) and your scores are matchpointed against other humans playing the same hands. You always get the most hcp at the table. The attraction of this format is that you have many more key decisions per hour than you do in a normal human game; you wind up declaring about 55% of the hands, and the hands go by pretty fast. As a result some people I know are total addicts…I’m a very mild but steady addict, playing 3-4 12-board sessions each week. Of course, the context of some decisions is very different from real bridge, where you wouldn’t (I hope) follow the rule of “never count on partner’s judgment.” You can read a long series of articles on Glen Ashton’s blog and some on Memphis Mojo’s. As far as tactics, I basically open a 14-18 notrump (a bit liberal on distribution) and 19-21 2NT. This is not only to hog the hand, but also because partner’s bidding is not quite as bad as in other auctions.

Partly because it is rare to have useful statistics on individuals in bridge, I got curious about breaking down my results and those of others. Bridgebase will give you a file with a month’s worth of results, and I wrote a parser to break these down by who declared. I might compile more statistics at some point. Here’s what my program outputs now, from being fed the last 3 months of my results (SD=standard deviation):


North declared 105 hands(24.36%): NS averaged 55.22%, SD = 23.15
East declared 41 hands(9.51%): NS averaged 53.77%, SD = 28.04
South declared 243 hands(56.38%): NS averaged 60.55%, SD = 26.31
West declared 37 hands(8.58%): NS averaged 61.80%, SD = 27.91
Pass declared 5 hands(1.16%): NS averaged 50.81%, SD = 10.13
Total of 431 deals, Average: 58.60% SD: 25.93
(Statistics exclude 1 average minuses.)

Comments: It’s not surprising I average better as declarer than as dummy. This doesn’t necessarily mean I should hog it even more – on a hand where I artificially made myself declarer by putting it in an inferior contract, my average would certainly be less than the 60.55% above. In fact, I’m quite satisfied with averaging 55% as dummy – this is a rare instance where we can say for sure that any advantage over the field represents solely bidding judgment! Good players who I’ve looked at tend to average 53-55% as dummy – this partly represents bidding to the right level, and partly that the field sometimes takes hand-hogging to excess. Finally, note that the swingiest hands are when I defend. The field doesn’t like to pass, so defending leads to some tops and bottoms. I defend a few percent more hands than most people I’ve looked at. (One reason is I avoid marginal takeout doubles, fearing partner’s insanity.) The results are acceptable, with a mp average on defense only about 1% less than my overall.

The highest MP average I’ve come across is Mark Lair’s. This could surprise you only if you didn’t know he plays the GiBs quite regularly. Here are his stats for a 2-month period:


North declared 558 hands(30.56%): NS averaged 55.07%, SD = 23.15
East declared 132 hands(7.23%): NS averaged 57.14%, SD = 25.97
South declared 1009 hands(55.26%): NS averaged 62.42%, SD = 24.68
West declared 120 hands(6.57%): NS averaged 55.20%, SD = 27.18
Pass declared 7 hands(0.38%): NS averaged 62.69%, SD = 16.47
Total of 1826 deals, Average: 59.32% SD: 24.72
(Statistics exclude 72 average minuses.)

I can certainly live with being just 0.7% worse than Mark Lair, at this odd but entertaining form of the game.

10 comments:

Dan said...

To be honest, I think the robot declarer is pretty darn good. But I also think that the robot declarer was built to play imps. And so you end up benefiting from hogging hands simply because you take proper views from a mp perspective. I always cringe when gib declares in a robot mp because I know it is going to take an esoteric safety play and leave me with 20% of the mps.

Memphis MOJO said...

Interesting post.

Jonathan Weinstein said...

The robot declarer can be pretty unpredictable, especially as to taking more or fewer tricks when playing at a different level of the same strain. Once I blackwooded and got him to 5S, where he just needed to pull trumps and claim 5 if they were 3-2 (they were.) He didn't pull trumps and went down 1. The field result was 4S-N, making 5! Some safety play.

Anonymous said...

Hey Jon,

Thanks for the interesting stats.

I am curious whether competitors who take a couple of minutes to plan a complex hand are doomed in these tourneys.

If you are really forced to play superfast, do you acquire any skill useful for regular tourney?

Also what are other big name experts playing with robots. No, I don't want to find out who is best expert via % in Best Hand tourney. Just participation in these increases my respect for pros.

Happy new year! Hope for more posts in 2010!

Regards,
Alex

Jonathan Weinstein said...

Alex,

Actually, the robot duplicates are very different from the robot races -- you can play quite slowly if you want -- 55 minutes for 12 hands, which is a lot when the other 3 players are robots. I actually play the interesting hands very slowly, knowing that for once I am not keeping any humans waiting. I take the opportunity to try as hard as ever not to play any wrong cards, never succeeding of course. I am almost always one of the last ones done, but with still 15 minutes or so on the clock!

The Pretender said...

I blame you, mojo, and ashton. I played 5 of these duplicates (8 boards, 25 minutes) this afternoon, and could have easily kept going.

The thing that astounds me the most is GIB's inability to stay within the description of its bid on the cc.

I can't judge the robot declarer because the robot defense is so atrociously bad.

The Pretender said...

I also just played a robot race, just finished out of the money.
I saw GIB commit this atrocity:

AJT9
K8
JT732
A8

Vul vs. NV
LHO opens 2NT (yea, I only had 18), RHO 3D xfer, LHO 3H, passed around GIB now bids 3S and goes for 1100.

How is this part of the program?

Jonathan Weinstein said...

Yeah, it regularly breaks its own system...this is sometimes, but not always because no bid fits its hand. I know that when it gets "off book" it uses "judgment," which means simulating some hands and guessing what to bid based on those. This can lead to some weird stuff for sure. I can't recall anything quite as bad as that 3S bid. I can only guess that because the auction is rare it only generated a very small sample of hands, which resulted in very random "judgment."

Anonymous said...

Hi Jonathan,

Where can I find the monthly robot Statistics ?

Jiang

applecor said...

I'm a big fan of robot duplicate on BBO, mostly for the reason Jonathan alludes to: in practice, you can play as fast or as slowly as you want, much more so than in a regular game, whether face-to-face or online.

The robot declarer plays some hands well, but many atrociously (even leaving aside the IMP/MP issue), particularly with a trump suit of 8 or fewer cards.

I have no stats to back this up, but I would guess that if you took two players A and B, where A knows and compensates for the robots' known eccentricities and B does not and plays "normal bridge", that A will average at least 5% higher than B. That and the distortions introduced by the "best hand" feature are, to me, the only bad things about these robot games. Otherwise, they are loads of fun and good tests of skill.

In answer to Alex, Les Bart and Tipton Golias are two other "name" players who play a fair amount of robot duplicate, and I saw Mike Passell there once.