Got It, Nate: Polling Is A Fraud. NOW You Tell Us.

 

Before we find out what happens in Alabama, I want to get this issue out there.

Yesterday I posted on Facebook in part,

“When did the polling profession go to Hell? Today a Fox poll shows Roy Moore 10 pts behind, and a local Alabama poll shows him 9 pts AHEAD. I’ve never seen anything like it. Moore has been edging ahead since last week, and now he’s losing again? Why bother with polling at all, if they can’t do better than this?”

This prompted two friends to send me to 538, the realm of alleged genius stat-head Nate Silver, who was pronounced the guru of election prognostication in 2012, and who became just another false prophet after failing as miserably in 2016 as everybody else. Silver posted an thorough, honest and disturbing explanation for the discrepancy, and one that didn’t include “Fox News is lying.”

It is worth reading; must reading, really. [Here it is.] The tipping point for me was this graph…

…to which Nate responded,

“Although releasing 10 different versions of the same poll may be overkill, it illustrates the extent to which polling can be an assumption-driven exercise…”

CAN be?

What I derive from Silver’s explanation is that polling doesn’t work any more, but the news media, politicians and pollsters want us to think it does.  There is no longer a reliable way to access a fair sampling of subjects, and the biases of pollsters create either unconscious or deliberate distortions in the poll results. This was always true to some extent [ DEWEY DEFEATS TRUMAN!] but now we are looking at a “science” that is about as reliable and trustworthy as astrology.

This intensifies my position regarding presidential popularity and performance polls, as well as the ever–popular “state of the nation” polls. I have always believed that they distort public opinion and public policy decisions, and are one more tool of unethical media manipulation of government. Any public official who cites a poll to justify his or her vote or position is instantly marked down in my Ethics Naughty List as a liar, a coward or a wimp. The recent polls about the Republican tax bills are a classic example. The public hasn’t read the bills . All they know is the anti-bill reporting they have read and the hysterical “This is the end of the United States!” op-eds that the irresponsible news media has been publishing. (That was a real headline.) Polls showing that a majority of the public “disapproves” of  bills they haven’t read, don’t understand and only know about through partisan reporting have no value to assessing the wisdom of passing a tax reform law at all. Nonetheless, the polls might sway a jelly-spined and feckless legislator, which is, in the end, the objective of the pollster.

Since January, polls have been weaponized as one more way to help undo the election. Popular Presidents can’t be impeached and convicted whether they deserve it or not (See: Bill Clinton), and it follows that if a President is unpopular enough, Congress might just get away with removing him in defiance of precedent and Constitutional requirements. This is what Maxine Waters is going around the country blathering to all who will listen. (That this would reduce the U.S. to a parliamentary form of government that our Founders specifically rejected is of no concern to “the resistance,” which is quite willing to destroy the nation to save it.) Thus we have been seeing polls, often as front page headlines and newscast top stories, braying that “Trump’s Approval Falls To Record Low!”

Such a headline intentionally ignores what Silver explains and should be obvious:  polls cannot be trusted to tell us anything useful or substantive. The pollsters can manipulate the findings, and do. The methodology is dubious. And “disapproval” is too vague a term in this context to mean anything.

Add to the growing list of traits that I am grudgingly forced to admire about Donald Trump: even though he cited polls constantly as a candidate, he is the first President in 50 years who does not appear to fear defying polls in order to do what he wants to do or promised to do. President Obama once said that he wouldn’t let polls govern his policies, but he pulled out of Iraq prematurely explicitly citing polls. Bill Clinton polled on whether he should tell the truth or not. George H.W. Bush used his high poll numbers to justify doing nothing for half his term.

We aren’t going to be able to get rid of polls, and I will accept the argument that they are better than nothing. To represent them as anything but highly unreliable pseudo-science, however, is irresponsible, and to use them to make policy decisions is incompetent.

And using them to try to bring down a President is indefensible.

8 Comments

Filed under "bias makes you stupid", Government & Politics, Journalism & Media, Leadership, Research and Scholarship, The Internet, This Helps Explain Why Trump Is President

8 responses to “Got It, Nate: Polling Is A Fraud. NOW You Tell Us.

  1. Other Bill

    I only wish this post had top billing in the NY Times, Washington Post, LA Times, etc.

  2. DaveL

    I’m afraid the problem hardly stops with polls. Simmons, Nelson, and Simonsohn have shown how decisions, in their case in psychological research, can be manipulated to find statistical significance where none exists. Other analyses suggest the problem extends to medical research as well. I can’t imagine the social sciences fare any better.

  3. Chris

    The public hasn’t read the bills .

    Well, in their defense, neither has Congress…

  4. valkygrrl

    AP called it for Jones.

  5. John Glass

    I too looked at 538 last night just to see what, if any, revisions were made to the modeling in the aftermath of his epic fail last fall. All he’s done is add more qualifications to slew of “studies” of dubious validity to the point of obfuscation. He’s covered himself so well he essentially says nothing. To use your Truman analogy, he’s become a one-handed statistician (“on the one the one hand, this poll says this, on the other hand that poll says that.”)

    But what I could glean from the overwritten analysis is that you cannot get a truly representative (read random) sample in today’s polling world. If the interviewee doesn’t have access to a cell phone and/or computer, and your model is based on this type of sampling, what do you have? Flip a coin. If he or she has call screener on a cell phone or landline and doesn’t answer unknown phone calls or resents the invasion of privacy and won’t answer the questions, how can you weight these unknowns?

    Most of the sample sizes are typically small (a thousand or so), so any sleight bias in methodology can have a big effect. The other thing that strikes me is that Silver is using what amounts to a kind of meta-analysis; that is, lumping together the results of a wide variety of “studies” – big or small, well-designed or poorly designed, etc. – and drawing inferences from them. This can be done, but it’s always tricky and may not be reliable. If you are looking for something in 10 or 20 instances, you’ll likely find it, though it may be by chance alone.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s