Tuesday, October 11, 2016

Replication

Andrew Gelman writes in Slate on the Replication Crisis: "Why Does the Replication Crisis Seem Worse in Psychology?"

The article begins with
"Last week, the replication crisis in psychology was pushed back into the news when Susan Fiske, a former president of the Association for Psychological Science, wrote a column in which she criticized “online vigilantes” on blogs, Twitter, and Facebook who have taken prominent work in social psychology to task. Fiske likened these “destructo-critics” to “methodological terrorists.” "

He then asks:
Why does psychology continue to dominate the news when it comes to discussion of the replication crisis?

Why not economics, which is more controversial and gets more space in the news media? Or medicine, which has higher stakes and a regular flow of well-publicized scandals?

He has 5 explanations (abbreviated here)
1. Sophistication: "..psychology is an inherently difficult field, studying constructs such as personality, intelligence, and motivation, which are undeniably important but which by their nature are “latent constructs” that cannot be measured directly."

2. Overconfidence deriving from research designs: When we talk about the replication crisis in psychology, we’re mostly talking about lab experiments and surveys. Either way, you get clean identification of comparisons, hence there’s an assumption that simple textbook methods can’t go wrong. 

3. Openness. ..In psychology, it’s relatively easy to get your hands on the data or at least to find mistakes in published work.

4. Involvement of some of prominent academics. But, in psychology, the replication crisis has engulfed Fiske, Roy BaumeisterJohn BarghCarol Dweck … these are leaders in their field. So there’s a legitimate feeling that the replication crisis strikes at the heart of psychology, or at least social psychology; it’s hard to dismiss it as a series of isolated incidents.

5. Everyone loves psychology: It’s often of general interest (hence all the press coverage, TED Talks, and so on) and accessible, both in its subject matter and its methods. 


What do you get when you put it together?
The strengths and weaknesses of the field of research psychology seemed to have combined to (a) encourage the publication and dissemination of lots of low-quality, unreplicable research, while (b) creating the conditions for this problem to be recognized, exposed, and discussed openly."

I actually always thought that there might be another reason (though admittedly I have not fully tested that theory with psychologists). But it seems in psychology, there is a huge bonus for having your own bias or model, and people mostly don't work on other people's experiments. In economics, on the other hand, it is perfectly fine to work on extensions of others' experiments and still have a great career. Luckily for me, for example, my paper with Lise Vesterlund was replicated many times, where most of these replications, and certainly almost all that are published, are in papers that have extensions where the baseline experiment was a big part of my experiment with Lise Vesterlund.

I feel that replications are common in experimental economics, and perhaps more so than in psychology, and so this may reduce the eagerness to publish results that feel not robust, and definitely may reduce the long successful careers some psychologists enjoyed before it was found out that their papers were fraudulent.

One way to think of this, is that there might be a real need to find out how many papers in economics are "replicated" where replications should be thought of in a very broad term. Then it could be interesting to compare this to psychology papers. And an important twist might be that a replication by the same author should perhaps not get the same weight as a replication by others.


2 comments:

  1. Kemenangan besar tiada batas hanya ada di PinoQQ ^^
    Daftarkan Sekarang di Partner >>> PELANGI QQ <<< Di Situs BANDARQ Terbaik hanya di PinoQQ

    Segera Daftarkan diri Anda di PINO QQ dan dapatkan Bonus yang sudah tersedia. Agen Poker Online Terpercaya dan Terbesar di Indonesia yang menggunakan Uang Asli.
    MINIMAL DEPOSIT & WITHDRAW Rp 20.000
    PINO QQ Menyediakan 8 Permainan yang bisa di mainkan hanya dengan 1 User ID,yaitu:
    * BANDARQ
    * SAKONG
    * POKER
    * DOMINO99
    * CAPSA SUSUN
    * ADU Q
    * BANDAR 66
    * BANDAR POKER
    Keunggulan PINO QQ :
    - PROSES DEPO & WD MUDAH TANPA RIBET
    - PROSES DEPO & WD TERCEPAT
    - KARTU-KARTU BERKUALITAS DISAJIKAN
    - CS RAMAH & INSPIRATIF SIAP MEMBANTU 24JAM
    - TIPS & TRIK MENJADI KEUNGGULAN SITUS INI
    - DAN TENTUNYA DEPOSIT YG TERJANGKAU BOS!!(MINIMAL DEPO & WD 20RB)

    Bagaimana cara mendaftar? SIMPEL bos ^^
    cukup kunjungi kami PINO QQ
    atau bisa juga hubungi kami melalui LiveChat dan BBM yang akan melayani Anda 24 jam nonstop.
    - LINE : pino_qq
    - PIN BB : DD02D468
    - WhatssApp : +855969787541
    - Instagram : mefylim7

    AGEN POKER ONLINE
    POKER ONLINE TERPERCAYA
    BANDAR Q ONLINE
    AGEN BANDAR Q
    TRIK AMPUH BERMAIN BANDAR Q
    TRIK AMPUH BERMAIN DOMINO 99
    AGEN DOMINO 99


    ReplyDelete
  2. CrownQQ Agen DominoQQ BandarQ dan Domino99 Online Terbesar

    Yuk Buruan ikutan bermain di website CrownQQ
    Sekarang CROWNQQ Memiliki Game terbaru Dan Ternama loh...

    9 permainan :
    => Poker
    => Bandar Poker
    => Domino99
    => BandarQ
    => AduQ
    => Sakong
    => Capsa Susun
    => Bandar 66
    => Perang Baccarat (NEW GAME)

    => Bonus Refferal 20%
    => Bonus Turn Over 0,5%
    => Minimal Depo 20.000
    => Minimal WD 20.000
    => 100% Member Asli
    => Pelayanan DP & WD 24 jam
    => Livechat Kami 24 Jam Online
    => Bisa Dimainkan Di Hp Android
    => Di Layani Dengan 5 Bank Terbaik
    => 1 User ID 9 Permainan Menarik

    Ayo gabung sekarang juga hanya dengan
    mengklick Agen BandarQ

    Link Resmi CrownQQ:
    - crownqq01.com
    - crownqq01.net
    - crownqq01.org

    Info Lebih lanjut Kunjungi :
    Website : CrownQQ
    Daftar CrownQQ : Poker Online
    Info CrownQQ : Kontakk
    Linktree : Agen Poker Online

    WHATSAPP : +855882357563
    Line : CS CROWNQQ
    Facebook : CrownQQ Official
    Kemenangan CrownQQ : Agen BandarQ

    ReplyDelete