How The Fact Checkers Cheat: A Case Study


A promising journalism watchdog website has come to my attention: Media Bias/Fact Check (MBFC News). On the “About” page we learn that it

…is an independent online media outlet. MBFC News is dedicated to educating the public on media bias and deceptive news practices. MBFC News’ aim is to inspire action and a rejection of overtly biased media. We want to return to an era of straight forward news reporting….MBFC News follows a strict methodology for determining the biases of sources….MBFC News also provides occasional fact checks, original articles on media bias and breaking/important news stories, especially as it relates to USA politics…

This is a relatively new site, launched in 2015.  It is unusual in that it aims to find both conservative and liberal bias alike. It will be interesting if it can keep to the middle of the road with all the crazy traffic coming at it from all directions. As Ethics Alarms can attest, this is harder than it looks.

The first example of MBFC News’s work (for me) is promising. As readers know, I distrust  factcheck sites and fact-checkers, as well as the periodic fact-checking exercises by sources like CNN. While sometimes a particular fact-checker may be fair and responsible, the same source can be overwhelmed with bias in another instance, and use dishonest or misleading means to discredit some disliked politician, usually a Republican. Some prominent fact-checkers. like Snopes and Politifact, are routinely biased and exist primarily to make progressives smile. Others, like the Washington Post’s “The Fact-Checker,” Glenn Kessler, have good days and bad days. At least Kessler tries; like many of his breed, however, he never learned what a lie is.

My favorite of the fact-checker services has long been, which also tries to be even-handed, and is more careful than Kessler. Thus I was impressed to see that when MBFC News set out to see how fair the fact-checkers were when they examined Donald Trump’s State of the Union Message, it examined the best. So did I, and was hunkering down as I prepared a post on what appears to be the Annenberg Foundation’s project’s capitulation to “the resistance.”

How I love it when someone else does my work for me, and does it well.  In an article that factchecks the factcheck (whew!) of the speech,  Karen O’Connor Rubsam writes,

“First, there are some global observations regarding the article. seems only to identify what they perceive as incorrect statements.  To be unbiased there should be some commentary on the entire address along with an overall assessment as to how much was “factual” versus “not-factual.”  A more thorough reporting of the entire address can be found here. Additionally, as shown below, introduces opinion and “biased words” in much of their fact-checking. Further, there appears to be some bias in how transferred the salient points from their analysis to the Summary bullet points.  Accurately reporting in the summary bullet points is important since many readers will just read the bullet points.”

Read it all at the site, which deserves the traffic, much as I would love to put up the whole thing. Two examples should suffice: when I read the analysis, these points, far from the worst,  caused me to conclude that the site had finally started playing typical factcheck games and gone over to the Dark Side, where bashing the President is deemed more important than being fair and truthful. (I promptly exiled it from the Ethics Alarms links):

  • “Trump said cutting the corporate tax rate will “increase average family income by more than $4,000.” This is a rosy, long-term estimate from White House economic advisers based on questionable assumptions.

Yecchh. This is classic unethical fact-checking: that others disagree with a long-term projection does not make a projection dishonest or wrong. If the factcheck establishment wants to take the position that all estimates and projections stated as what “will” happen are misleading, fine: let them state that as a principle. But every bill has been and is announced by a claim what it “will” accomplish, and the fact that opponents are dubious doesn’t make that statement untrue when it is made. Every climate change projection is questionable in exactly the same way. Presumably intelligent people don’t have to be told that predictions and projections are always guesses, and when a President says something “will” happen, it means “unless it doesn’t.”

  • “The President wrongly said that the U.S. is “an exporter of energy to the world.” The Energy Information Administration estimates the U.S. won’t be a net exporter of energy until 2026.”

What? The President said that the U.S. “an exporter of energy to the world,” and it is an exporter of energy to the world. If he had said, which he didn’t, the U.S. is “a net exporter of energy to the world,” then this critique would be fair. He didn’t.


Pointer: Michael McMurphy




13 thoughts on “How The Fact Checkers Cheat: A Case Study

  1. The “Annenberg Foundation?”

    That the same entity that was…um…responsible for the Chicago Annenberg Challenge where, with the help of a young up-n-comer (Barack Obama) and one William Ayers, helped piss away a half a billion (that’s with a “B!”) dollars?

    From the bedrock Conservative LA Times:

    ”But today (04/07/1998), as the Annenberg Challenge proceeds into its fifth and final year, all hopes have diminished. The promised improvements have not been realized. (bolds mine)

    Tommy Steyer should take note, but he won’t.

  2. A left-left friend brought this site to my attention on Facebook, with a note that she expects I would like this fact-checking site. I do, and I give my own ethics hero award to her for posting it on Facebook. Unexpected, given that most of her posts are the strongly anti-Trump “shares”.

  3. A point to your meme. I don’t read vox nor link to vox and the only articles I notice from there are ones you post. Since the Facebook crackdown, I have seen ads for Vox at least 3 times. This could be coincidence, but I’m becoming more of a skeptic.

    • I remember you writing a little bit on this issue.
      If Trump is wrong on your particular issue, he is either a. lying to deceive or b. because he doesn’t any better. Both results are unethical.

      If you didn’t get a chance to read it, make sure you examine what the posted article said regarding immigrant visa program. Here it is.

      Summary Fact Check #11

      Trump wrongly said the Diversity Immigrant Visa Program “hands out green cards without any regard for skill … or the safety of the American people.” There are both education or work experience requirements, and a background check for all who are selected.
      Actual quote – “The third pillar ends the visa lottery, a program that randomly hands out green cards without any regard for skill, merit, or the safety of American people. It’s time to begin moving towards a merit-based immigration system, one that admits people who are skilled, who want to work, who will contribute to our society, and who will love and respect our country.”

      What they say “in the detail” – “While calling for an end to the Diversity Immigrant Visa Program, Trump inaccurately described it as “a program that randomly hands out green cards without any regard for skill, merit, or the safety of American people.” In order to be eligible for the lottery, applicants must have a high school education or two years of work experience, and if selected they must go through a vigorous safety background check, the same as all legal immigrants. The Diversity Immigrant Visa Program uses a computer lottery system to randomly issue up to 50,000 immigrant visas each year — from the millions who apply annually from countries with low rates of immigration to the United States. Trump has regularly botched the description of the program, mistakenly claiming that other countries are gaming the system to send the U.S. their “worst.” There’s no evidence of that. It’s true that applicants do not need to meet the requirements for employment-based immigration visas, but Trump went too far with the claim that the diversity visa program hands out green cards “without any regard for skill … or the safety of American people.” Applicants must demonstrate that they have a high school education or its equivalent or “two years of work experience within the past five years in an occupation that requires at least two years of training or experience to perform.” In addition, if selected, applicants have to go through a background security vetting process. “National security is our top priority when adjudicating visa applications,” a State Department official told us. “Every prospective traveler to the United States undergoes extensive security screening. No visa can be issued unless all concerns raised by the screening are fully resolved.” Data from the Department of Homeland Security’s 2015 Yearbook of Immigration Statistics indicate that 32 percent of those who came through the DV program in 2015 were employed in management, professional and related occupations — a lower percentage than those who came via employment-based preferences (41 percent), but far higher than the percentage among those who came via family-sponsored preferences (12 percent) or among those who were granted green cards because they were immediate relatives of U.S. citizens (9 percent). The report also indicates that about 1 percent of DV immigrants were listed as unemployed. That’s far lower than the percentage of unemployed people among all green card recipients (5.1 percent).”

      Counter-point – Again, is more forthcoming in their analysis than the summary bullet point. The summary point merely states, that “There are both education or work experience requirements, and a background check for all who are selected” whereas the analysis states that “It’s true that applicants do not need to meet the requirements for employment-based immigration visas, but Trump went too far with the claim that the diversity visa program hands out green cards “without any regard for skill … or the safety of American people.” also fails to provide the context of Trump’s statement – that being the concerns being raised regarding the Diversity Immigrant Visa given the terrorist arrested for the deadly truck rampage through a New York City bike path entered the United States under this visa program.

  4. All anyone really needs to know about Media Bias/Fact Check is that it rates PolitiFact as a “Least-biased” source and rates it highly for factual content. It actually says PolitiFact is the “gold standard” for fact-checking.

    That’s just silly. The observations about ring true, but Fact Check, though left-leaning, is head and shoulders better than PolitiFact.

    The bias rating methodology at MB/FC is subjective. As such, it’s an effective lure for traffic thanks to confirmation bias. People often use MB/FC ratings as a type of proof that PolitiFact is a reliable and unbiased fact checker.

    MB/FC is also guilty of a number of embarrassing missteps, which I won’t go into here. A bit of Googling ought to bring those to light for those who are interested.

    If a worthwhile bias rating site exists it’s probably the AllSides project.

      • I am the Bryan W. White who wrote dozens of criticisms of PolitiFact at the Sublime Bloviations blog. I also helped create PolitiFact Bias and wrote/write most of its content. And after that, seeing the need for an improved fact-checking model, I created Zebra Fact Check (thank you, Phlinn, for the compliment).

        Subjective rating scales are one of the top problems that make media fact-checking awful. Media Bias/Fact Check uses a subjective rating scale. We have more than enough of those already.

          • It was a reasonable question whatever your motive, in my opinion (I made no assumptions about your motive).

            Having read my work you may be aware that I’m fine with people adjusting their views of my work because of my conservatism. I simply ask that it’s done on a reasonable basis. Find the problem in what I write, not in who I am ideologically.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.