BLOG

Chainsaw Review of How to Lie with Statistics, Part I

Darrell Huff’s Satirical, Statistical Classic Offers All the Tools You Can Use to Create Convincing, and Arguably True, Untruths

Long before “truthiness” was a thing, author Darrell Huff coined the word “statisticulation,” which he defined as “misinforming people by the use of statistical manipulation.”

Cover to how to lie with statistics

How to Lie with Statistics

It’s not exactly lying, and therein lays the challenge for professional statisticulators everywhere: How to use the cold, hard facts to tell a big, fat lie.

Written in 1954, only the examples offered by Huff’s bestseller are outdated, but the strategies and statisticulations discussed are still in use all around you.

Whether you want to see through all these statistical shenanigans, or simply use them to promote your own falsehoods, agendas, ideologies and products, this fine little book remains your classic guide to the Misinformation Age.

The Sample with the Built-In Bias

Statisticulation begins with sampling, and as Huff writes, “the result of a sampling study is no better than the sample it is based upon.”

Thanks to reliable polling organizations like Pew and Gallup, most of us automatically assume that scientific samples must be somewhat valid, or else the study wouldn’t be published. Nothing could be further from the truth.

You already know that surveys of Drudge Report readers, Wholefoods shoppers, and people who still use landline telephones would give radically different—and largely predictable—answers to questions like, “Do you think Obama has been a good President?” or “Do you believe the use of deadly force by police is usually justified?”

There are subtler ways to skew the poll as well. The race, sex, or appearance of pollsters makes a statistically significant, and usually predictable, difference. A black pollster and a white pollster asking these same two questions at the same suburban mall would probably get very different results.

This particular tactic is commonly used when asking people to self-report their eating and exercise habits; an overweight pollster gets different answers than a very fit one.

In fact, any poll that asks people to self-report activities with cultural or ethical baggage—drug use, church attendance, off-line reading habits— generally gets different results than you would be actually observing their behavior.

Finally, any poll that uses an “opt-in” or “opt-out” survey method (this was Huff’s biggest complaint and is now considered unethical) has probably been deliberately skewed.

What Is Average?

When Mitt Romney was running for President of the United States, one of his many gaffs was suggesting a “middle class income” was US$250,000 per year. Everyone else pointed out that the average US income is about $55,000 per year.

But, Romney was absolutely correct. The mean family income in the United States is about $250,000. The median income is around $50,000. Both words are synonyms for “average.”

Had Romney read Huff’s book, he’d have known the difference between mean and median, and might be President right now.

So, why the huge difference? To get the mean, or arithmetic average, add up all family incomes and divide by the number of families. In the USA, that’s around $250,000 per year. For the median, you graph US family incomes on a bell curve and use the top of the curve; that works out to around $55,000 per year.

If you wanted to be really tricky, you could use the mode, which the most frequent figure a series of numbers. That would be about $30,000, a dual-income family making minimum wage.

(As an aside, the definition of “family” is also a little bit squirrely; calculating the income for individual workers is a much more precise way to give this information. It’s rarely used, however, because it’s more difficult to skew for political purposes.)

Despite Romney’s genuine confusion, the mean and median are deliberately, rather than accidentally, chosen for specific purposes. The mean is more effective when you’re trying to sell a home in a “posh” neighborhood, or describe a company’s average salary by including the CEO’s $26 million annual package. The median is better when reporting economic news or debating tax increases.

You can also use this type of data to skew things like average height, weight, IQ scores and disease risks in a given population, among many other things. Be creative, the possibilities are almost endless.

Much Ado About Practically Nothing

Do you believe in global warming? If you’re running a 24-hour news network that markets itself by cultivating a deliberate political bias, the answer is, “The same thing that our viewers believe.”

What better way to lead into a theatrical [read: screamy] discussion of global warming than with a weather report that supports your viewers’ established opinions? Unfortunately, weather isn’t climate, so this will only work sometimes, when it’s hotter or colder than average.

Here’s the trick—it’s always hotter or colder than average.

Averages are, as noted above, an important part of every professional statisticulator’s toolbox. For example, only about 5% of babies are born on their due date. A small sample size (say, five babies) guarantees that an alarming number of children are being born early or late. Is that a trend? A cause for concern? A result of global warming?

That depends on what point you’re trying to make. No matter what, however, the average—being born on the due date—is the least likely outcome.

The same goes for the weather.

For example, a spring day in Albuquerque can see nighttime temperatures that dip below freezing and daytime temperatures soaring into the 90s (35°C). The average April temperature in Albuquerque is 69°F (21°C). At what point is it 69°F? For about ten minutes, twice per day, around 11:14 a.m. and 7:43 p.m.

Chainblog_statblog_BrBa
Which is why we recommend dressing in layers for the high desert springtime, just in case the situation gets too hot. Photo courtesy of AMC

This is why both FOX and MSNBC can use Albuquerque’s April average to support or undermine global warming. The FOX reporter can start in the morning: “A thin dusting of snow remains after a brutally cold night here in Duke City, where normal springtime temperatures average 69°F.”

The MSNBC correspondent could file her report in late afternoon: “It’s an outlandish 93°F this afternoon at the Albuquerque Sunport, a devastating 22°F higher than average temperatures recorded here over the last century.”

Either way, it’s enough to make even casual news viewers consider turning to drugs.

The Little Figures That Aren’t There

Huff’s outdated example is a toothpaste company, which cheerfully promised, back in the early 1950s, that its users got 23 percent fewer cavities than people brushing with other brands.

“Yet if you are not outstandingly gullible or optimistic,” Huff continues, “you will recall from experience that one toothpaste is seldom much better than any other.”

A quick look at the methodology revealed that the study’s sample size was a scant twelve people. The 23 percent difference worked out to a single cavity between them.

This exact type of statisticulation has since fueled an entire industry: the diet media industry. Did you hear, for example, that drinking one glass of wine was equivalent to an hour at the gym? Or that superfoods can help heal cancer?

So did actual scientist John Bohannon. That why he, and io9.com, decided to play a prank on the diet media, by “proving” that eating chocolate helps you lose weight.

Sample size? 15 people. Is that normal for a diet research study? Yes. Did they lose weight? Yes. Well, that’s science, right? Yes. Bad science.

“It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded,” explained Bohannon. “If you measure a large number of things about a small number of people, you are almost guaranteed to get a ‘statistically significant’ result…that chocolate ‘improves sleep’ or ‘lowers blood pressure.’”

Or leads to weight loss.

Shape, Men’s Health, Daily Star, Prevention, Huffington Post, Cosmopolitan, and the Examiner were just a few of the publications blasting out the good news that eating chocolate was going to get rid of that weight without diet or exercise.

The moral of this story (and pretty much the entire book, but we’ll get to that in Part II) is that if it sounds too good to be true, it probably is.

2016-10-21T11:03:09-05:00 By |Categories: Communication, Psychology, Reviews|Tags: , |