Six Ethics Lessons As Bill James Falls Into His Own Research Trap

Baseball philosopher, iconoclast and analyst Bill James is one of my heroes for his amazing ability to look past conventional wisdom with an open mind. Beginning as essentially a self-published pamphleteer writing out of his basement, James’ counter-traditional explorations of baseball statistics eventually changed how baseball was watched, assessed, scouted and played, simply on the strength of Bill’s  ideas and his facility in explaining them.

His talents could be used in many other fields–James has recently branched out into examining famous unsolved murders—but it is also true that many of the ideas he has developed in relation to baseball have wider applications. For example, James was the source of the concept of “signature significance,” which is a staple here at Ethics Alarms.

His writing also taught me that bias makes us stupid, and about the insidious power of rationalizations.  Many of James’s observations seemed intrinsically obvious once he made and explained them, and the fact that  baseball executives, writers and players could have been so wrong about their own game for so long seemed incomprehensible. But the reasons were what they always are, in all fields. People are biased toward what they have always believed —confirmation bias–and the “It’s always been this way” variation on the most powerful rationalization of them all, “Everybody does it”  breeds blindness and  intellectual laziness.

It was Bill James, for example, who explained that how often a player got on base, a percentage that included bases on balls and being hit by pitches, was more important in judging a players’ offensive value than his batting average, which only counted hits. Now, literally every baseball team accepts this, but in the late 1970’s, when James started writing about it, almost nobody did. Of course, everyone assumed, a .300 hitter was better than a .250 hitter! Then James carefully, logically, showed how a .300 hitter with just a .325 on-base-percentage scored fewer runs and made more outs than a .250 hitter with a .370 OBP, meaning that the player with the lower average was better.

This is all a prelude to using Bill as a cautionary tale about how even the most brilliant people can fall into traps that end up resulting in misleading information.

In James’ current annual publication, the Bill James Handbook, he has a long essay about who the public wants to see admitted to the Major League Baseball Hall of Fame. He concludes, after a laborious study that he devised,  that the 14 players the public most wants to see in the Hall include Barry Bonds, Pete Rose, Roger Clemens, Manny Ramirez, Joe Jackson and Alex Rodriguez, with Bonds being the most favored of all. This, Bill says, is who “the public” most wants to go in; the four most notorious steroid cheats, and the two greats banned for gambling

Wrong. Very wrong, in fact. Bill’s pool for his study, which was otherwise well designed, was Twitter, in part because the platform has a neat polling application. Twitter, however is not a fair representation of baseball fans, any more than it is a fair representation of the country, or voters, or, though I hate to say it, smart people. Twitter, as hasbeen cited in many forums and many studies, is disproportionately  young, libertarian and progressive, which makes it especially ill-suited for a baseball survey, since baseball fans are older and more conservative than the general population.

Young progressives and libertarians are laissez-faire, to be nice about it, regarding drugs, illicit drug use, and cheating, since studies show that they probably have cheated themselves. Those under 30 barely remember the steroid scandal of the 90s, and Pete Rose and Joe Jackson are just names to them. Other polls, like this one in 2017, found that the public did not want steroid cheats in the Hall, and by a pretty large margin. Another poll finds a narrow majority wants Pete Rose allowed into the Hall, but that his support is falling even as legalized gambling is spreading.

Bill’s study had a misleading result, and one that could enhance the chances of these bad apples being eventually placed in the Hall of Fame pie, making it nauseating for anyone who understands ethics.

How could James make such a basic research error? I don’t know: he tried to rationalize his choice of Twitter in his introduction, joking about the imperfections of any pool and essentially falling back on “it’s not perfect, but what is?” He knows better. Long ago James wrote about how big studies were inherently unreliable because the researchers spent so much time on them that they could never bear to admit, “Well, that was a bust.” I think Bill realized he had chosen a bad medium for his study as soon as he saw that Bonds was the runaway Hall of Fame favorite. He had an article to write, though, a publishing deadline, and did exactly what he accused other researchers of doing.

The ethics lessons here are…

1. Even brilliant people are wrong sometimes.

2. Hubris stalks us all, even the most modest and perceptive of us.  Bill is so accustomed to being right when he challenges convention that he is a likely victim  of Rationalization #14, Self-Validating Virtue. “Hey, I’m Bill James” he thinks. “I’m usually right when everyone thinks I’m wrong, and I thought up this study, so it must be right!”

3. It’s easier to show why others were wrong than to avoid their mistakes.

4. Icons, heroes, role models and acknowledged geniuses in a field have a special obligation to be extra careful, because their opinions and conclusions carry special power to persuade and mislead.

5. When you hear those ethics alarms ringing, however faintly, pay attention, and

6. Never trust Twitter. Ever.


Please use this Twitter link if you share this on Facebook, as I would love you to do:

3 thoughts on “Six Ethics Lessons As Bill James Falls Into His Own Research Trap

  1. 6. Never trust Twitter. Ever.

    Please use this Twitter link if you share this on Facebook.

    That is, trust Twitter just a hair’s breadth more than Facebook*, but not any more.
    (*This does not imply that any measure of trust is due to Facebook!)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.