Tuesday, October 11, 2016

Replication

Andrew Gelman writes in Slate on the Replication Crisis: "Why Does the Replication Crisis Seem Worse in Psychology?"

The article begins with
"Last week, the replication crisis in psychology was pushed back into the news when Susan Fiske, a former president of the Association for Psychological Science, wrote a column in which she criticized “online vigilantes” on blogs, Twitter, and Facebook who have taken prominent work in social psychology to task. Fiske likened these “destructo-critics” to “methodological terrorists.” "

He then asks:
Why does psychology continue to dominate the news when it comes to discussion of the replication crisis?

Why not economics, which is more controversial and gets more space in the news media? Or medicine, which has higher stakes and a regular flow of well-publicized scandals?

He has 5 explanations (abbreviated here)
1. Sophistication: "..psychology is an inherently difficult field, studying constructs such as personality, intelligence, and motivation, which are undeniably important but which by their nature are “latent constructs” that cannot be measured directly."

2. Overconfidence deriving from research designs: When we talk about the replication crisis in psychology, we’re mostly talking about lab experiments and surveys. Either way, you get clean identification of comparisons, hence there’s an assumption that simple textbook methods can’t go wrong. 

3. Openness. ..In psychology, it’s relatively easy to get your hands on the data or at least to find mistakes in published work.

4. Involvement of some of prominent academics. But, in psychology, the replication crisis has engulfed Fiske, Roy BaumeisterJohn BarghCarol Dweck … these are leaders in their field. So there’s a legitimate feeling that the replication crisis strikes at the heart of psychology, or at least social psychology; it’s hard to dismiss it as a series of isolated incidents.

5. Everyone loves psychology: It’s often of general interest (hence all the press coverage, TED Talks, and so on) and accessible, both in its subject matter and its methods. 


What do you get when you put it together?
The strengths and weaknesses of the field of research psychology seemed to have combined to (a) encourage the publication and dissemination of lots of low-quality, unreplicable research, while (b) creating the conditions for this problem to be recognized, exposed, and discussed openly."

I actually always thought that there might be another reason (though admittedly I have not fully tested that theory with psychologists). But it seems in psychology, there is a huge bonus for having your own bias or model, and people mostly don't work on other people's experiments. In economics, on the other hand, it is perfectly fine to work on extensions of others' experiments and still have a great career. Luckily for me, for example, my paper with Lise Vesterlund was replicated many times, where most of these replications, and certainly almost all that are published, are in papers that have extensions where the baseline experiment was a big part of my experiment with Lise Vesterlund.

I feel that replications are common in experimental economics, and perhaps more so than in psychology, and so this may reduce the eagerness to publish results that feel not robust, and definitely may reduce the long successful careers some psychologists enjoyed before it was found out that their papers were fraudulent.

One way to think of this, is that there might be a real need to find out how many papers in economics are "replicated" where replications should be thought of in a very broad term. Then it could be interesting to compare this to psychology papers. And an important twist might be that a replication by the same author should perhaps not get the same weight as a replication by others.


2 comments:

  1. JOIN SEKARANG Kamu Ingin Merasakan Sensasi Kartu Terbaik?
    Daftarkan Sekarang DI Partner >>>Poker Pelangi<<< Di Situs BandarQ Terbaik Hanya Di MarioQQ
    MarioQQ Memberikan Fasilitas Terbaik Untuk Bermain Kartu Dapatkan Sekarang Juga!
    Kesempatan Bermain Bermodalkan 10rb Saja!
    Motto Logo MarioQQ Memberikan Kemenangan Besar Hanya Dengan Modal Minimal Rasakan Sekarang!
    8 Games Dalam 1 User ID
    -BandarQ
    -AduQ
    -Domino99
    -Capsa Susun
    -Sakong
    -Poker
    -Bandar Poker
    -Bandar 66

    Tunggu Apa Lagi Daftarkan Sekarang !

    Rasakan Kemenangan Bermain Dengan Modal 10rb Saja!
    Hubungi Kami!
    -BBM : DC91DE02
    -WA : +855967975460
    -Line : mari0qq
    -FB : MarioqqLink

    Tunjukan Kalo Kamu Adalah Yang Terbaik!
    >>>Poker Pelangi<<<
    Kunjungi Juga Situs Kontes SEO
    >>>CCSEO<<<
    >>>CC SEO<<<
    >>>CCSEO<<<
    >>>CC SEO<<<
    >>>MarioQQ SEO<<<
    Kunjungi Juga Situs Berita Terkini!
    >>>Cerita Rakyat Indonesia<<<
    Kunjungi Juga Situs Berita Viral !
    >>>Kerang Blogger<<<
    >>>Kakek Gokil<<<
    Kunjungi Juga Situs Dewasa!
    >>>Kamus Dewasa<<<
    Agen Poker Online Terpercaya !
    >>>Agen Situs Poker<<<
    >>>Agen Poker Online Terpercaya<<<

    ReplyDelete
  2. Kemenangan besar tiada batas hanya ada di PinoQQ ^^
    Daftarkan Sekarang di Partner >>> PELANGI QQ <<< Di Situs BANDARQ Terbaik hanya di PinoQQ

    Segera Daftarkan diri Anda di PINO QQ dan dapatkan Bonus yang sudah tersedia. Agen Poker Online Terpercaya dan Terbesar di Indonesia yang menggunakan Uang Asli.
    MINIMAL DEPOSIT & WITHDRAW Rp 20.000
    PINO QQ Menyediakan 8 Permainan yang bisa di mainkan hanya dengan 1 User ID,yaitu:
    * BANDARQ
    * SAKONG
    * POKER
    * DOMINO99
    * CAPSA SUSUN
    * ADU Q
    * BANDAR 66
    * BANDAR POKER
    Keunggulan PINO QQ :
    - PROSES DEPO & WD MUDAH TANPA RIBET
    - PROSES DEPO & WD TERCEPAT
    - KARTU-KARTU BERKUALITAS DISAJIKAN
    - CS RAMAH & INSPIRATIF SIAP MEMBANTU 24JAM
    - TIPS & TRIK MENJADI KEUNGGULAN SITUS INI
    - DAN TENTUNYA DEPOSIT YG TERJANGKAU BOS!!(MINIMAL DEPO & WD 20RB)

    Bagaimana cara mendaftar? SIMPEL bos ^^
    cukup kunjungi kami PINO QQ
    atau bisa juga hubungi kami melalui LiveChat dan BBM yang akan melayani Anda 24 jam nonstop.
    - LINE : pino_qq
    - PIN BB : DD02D468
    - WhatssApp : +855969787541
    - Instagram : mefylim7

    AGEN POKER ONLINE
    POKER ONLINE TERPERCAYA
    BANDAR Q ONLINE
    AGEN BANDAR Q
    TRIK AMPUH BERMAIN BANDAR Q
    TRIK AMPUH BERMAIN DOMINO 99
    AGEN DOMINO 99


    ReplyDelete