Andrew Gelman writes in Slate on the Replication Crisis: "
Why Does the Replication Crisis Seem Worse in Psychology?"
The article begins with
"Last week, the replication crisis in psychology was pushed back into the news when Susan Fiske, a former president of the Association for Psychological Science, wrote a column in which she criticized “online vigilantes” on blogs, Twitter, and Facebook who have taken prominent work in social psychology to task. Fiske likened these “destructo-critics” to “methodological terrorists.” "
He then asks:
Why does psychology continue to dominate the news when it comes to discussion of the replication crisis?
Why not economics, which is more controversial and gets more space in the news media? Or medicine, which has higher stakes and a regular flow of well-publicized scandals?
He has 5 explanations (abbreviated here)
1. Sophistication: "..psychology is an inherently difficult field, studying constructs such as personality, intelligence, and motivation, which are undeniably important but which by their nature are “latent constructs” that cannot be measured directly."
2. Overconfidence deriving from research designs: When we talk about the replication crisis in psychology, we’re mostly talking about lab experiments and surveys. Either way, you get clean identification of comparisons, hence there’s an assumption that simple textbook methods can’t go wrong.
3. Openness. ..In psychology, it’s relatively easy to get your hands on the data or at least to find mistakes in published work.
4. Involvement of some of prominent academics. But, in psychology, the replication crisis has engulfed Fiske, Roy Baumeister, John Bargh, Carol Dweck … these are leaders in their field. So there’s a legitimate feeling that the replication crisis strikes at the heart of psychology, or at least social psychology; it’s hard to dismiss it as a series of isolated incidents.
5. Everyone loves psychology: It’s often of general interest (hence all the press coverage, TED Talks, and so on) and accessible, both in its subject matter and its methods.
What do you get when you put it together?
The strengths and weaknesses of the field of research psychology seemed to have combined to (a) encourage the publication and dissemination of lots of low-quality, unreplicable research, while (b) creating the conditions for this problem to be recognized, exposed, and discussed openly."
I actually always thought that there might be another reason (though admittedly I have not fully tested that theory with psychologists). But it seems in psychology, there is a huge bonus for having your own bias or model, and people mostly don't work on other people's experiments. In economics, on the other hand, it is perfectly fine to work on extensions of others' experiments and still have a great career. Luckily for me, for example, my paper with Lise Vesterlund was replicated many times, where most of these replications, and certainly almost all that are published, are in papers that have extensions where the baseline experiment was a big part of my experiment with Lise Vesterlund.
I feel that replications are common in experimental economics, and perhaps more so than in psychology, and so this may reduce the eagerness to publish results that feel not robust, and definitely may reduce the long successful careers some psychologists enjoyed before it was found out that their papers were fraudulent.
One way to think of this, is that there might be a real need to find out how many papers in economics are "replicated" where replications should be thought of in a very broad term. Then it could be interesting to compare this to psychology papers. And an important twist might be that a replication by the same author should perhaps not get the same weight as a replication by others.
