Sensationalist Science

How Pride Corrupts Peer-Review

Well if scientists say it, it must be true, right? It turns out, not always. Why we shouldn’t believe every new study, next on BreakPoint.

Listen Now | Download

John Stonestreet

Every dubious claim published online or in the grocery store checkout aisle comes with the backing of “a new study.” Political pundits swing social science findings around like clubs, and who hasn’t thumbed through those magazines that promise “Ten Scientifically-Proven Ways to Make Yourself Irresistible to the Opposite Sex”? We stamp the seal of scientific approval on just about everything these days. But have you noticed how often science contradicts itself?

Take the highly-politicized research on how conservative states have higher divorce rates than liberal ones. The conclusion that many drew from this was that the so-called “red state” family model of abstinence, monogamy, and big families is inferior to the “blue state” model of sexual freedom and 2.1 children. Well, more in-depth research this summer debunked those claims, pointing out the critical fact that red states have more divorces, well because they have more marriages. But you can be sure that myth will live on because it’s often the politically-useful headlines, and not the facts, that end up mattering the most.

Still, cracks are beginning to show in sensationalist science, and ironically, the latest blow comes in the form of—you guessed it—a new scientific study.

daily_commentary_09_22_15Writing in The New York Times, Benedict Carey describes a University of Virginia-led effort to reproduce the findings of 100 key psychological studies published in top journals. Over 250 researchers chose some of the most often cited findings in their field and tried to replicate the results with their own experiments. The outcomes, published in the journal, “Science,” weren’t pretty. Of the 100 studies tested, 60 did not yield the results their authors reported. In other words, the findings couldn’t live up to a basic requirement of science—repeatability. It’s a revelation Carey says confirms many scientists’ worst fears.

“The vetted studies,” he explained, “were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory.” And the fact that so many studies were called into question makes one question the work of therapists and educators who relied on that research to do their jobs.

Now what’s behind this embarrassing revelation? Why are so many scientists apparently exaggerating and misinterpreting their findings? Carey points to what the scientists themselves describe as “a hypercompetitive culture across science that favors novel, sexy results and provides little incentive for researchers to replicate the findings of others, or for journals to publish studies that fail to find a splashy result.”

In other words, sensationalist science is its own undoing. But there’s more to it. Norbert Schwarz, a psychology professor at the University of Southern California, tells the Times that many senior researchers bristle at the thought of a younger, less experienced scientist critiquing their work. “There’s no doubt,” he said, “that replication is important, but it’s often just [seen as] an attack, a vigilante exercise.”

In other words, the real flaw in a lot of research isn’t technical or methodological. It’s just old-fashioned human pride. And it’s not restricted to psychology or the social sciences. Dr. John Ioannidis, director of Meta-Research at Stanford, hints that the peer-review climate could be even more toxic in other fields, like cell biology, economics, neuroscience, clinical medicine, and animal research, calling the reliability of science itself into question.

So what should Christians make of all this? Well first, it’s not so bad if pop-science loses some luster. Our culture sets scientists up on a pedestal, and as physicist Stephen Hawking demonstrated when he oddly declared that science has replaced philosophy, it can go to their heads. Second, this should remind us that science doesn’t have all the answers. In fact, the more political, ideological, or lucrative the stakes, the more likely those “splashy results” are to be fish stories. And Christians know the reason: because inside every white lab coat and bow tie is a fallible human being, just like you and me.

Further Reading and Information

Sensationalist Science: How Pride Corrupts Peer-Review
Honest criticism is important, no matter the field of endeavor.  We use science as an instrument to search for answers and solutions, but instruments are not infallible because they are wielded by imperfect human beings. It's a fact that should keep all of us, scientists included, walking in humility.



Many Psychology Findings Not as Strong as Claimed, Study Says
Benedict Carey | New York Times | August 27, 2015

Skeptical Science: Questionable Data Plagues Scientific Research
Eric Metaxas | BreakPoint.org | June 1, 2015

Scientism Exposed: Volcanoes, Computers, and We Ain’t as Smart as We Think
John Stonestreet | BreakPoint.org | April 29, 2015


Old news
This is not news. John Ioannidis has spent a career critiquing the quality of published research. See:

Why Most Published Research Findings Are False

Unfortunately Christians tend to be just as selective in their regard, or disregard, for scientific data. I won't mention specific examples because who wants to light a firestorm?

So will the next column be on how pride corrupts Biblical exegesis, church politics, or the genesis of political opinions?
One missing factor.....
There is another peculiarly human factor in the scientific community's hostility toward the reviewing of previous studies: MONEY! Studies need grants, and sexy results get more grants, and studies that seek to re-examin previous findings that produced sexy results have a hard time getting grants because grant committees are chaired by scientists who are famous for getting the sexy results with those previous studies! For example: do you think any scientist, no matter how reputable, could get a grant for a study that might disprove human-caused climate change? Good luck with that!
Follow the Money
FACT: Science research projects cost money. That means many scientist are in competition to get that cash.
So they need to present compelling arguments as to why their research project is the one to fund. In that process they better promise conclusions that will be news worthy interesting facts.

It is easy to rationalize twisting the truth and being dishonest when you need money for something you think is important. Especially when others are doing the same thing.