Pre-registered replications

Quick jump to sections: BackgroundProject summary / Get involved: Resources / Publications & preprints / Media attention / Our team  / Completed replications  / Planned for 2019-20 / Registered Reports



In 2016, following recent developments in psychological science (the so-called “replication / reproducibility crisis”) and gaining my academic independence, I decided to make major changes to my research agenda to prioritize pre-registered replications and pre-registered meta-analyses and focus on the realm of judgment and decision making. The aim was to revisit research findings I once took for granted and re-establish the foundations on which I hope to build my research. I, therefore, decided that all my teaching and mentoring work with guided thesis students will involve either pre-registered replications or pre-registered meta-analyses, atleast as a first step, to examine the classics in the field.

In 2017 I guided 3 masters students at Maastricht University to pre-test this realignment. It far exceeded my expectations. We completed 3 pre-registered replication, 3 pre-registered meta-analyses, and one review paper summarizing the insights gained. Once joining HKU, in Dec 2017, I decided to scale up and mass-mobilize HKU’s undergraduate students and lead a massive pre-registered replication effort. In the first year, two semesters, of running this project, we’ve successfully completed 45 replication projects, making this one of the largest replication efforts in social-psychology. For each of the replication projects, we have full pre-registrations, data/code, and all written up in APA style submission ready student reports. In the second semester, most of the replications also included extensions with interesting contributions and insights.

I will continue running this in academic year 2019-20 with 20+ new replications+extensions. If any of this is of interest to you – lots of ways to join in. I am looking for interested early career researchers to join us, see more info on that below.


Project summary

WARNING: Preliminary student summarized findings, need to be rechecked and verified.

I summarized findings in a poster presented in summer 2019

Portrait version for SIPS2019:

2019 mass pre registered replication project status update portrait poster

Download the poster: PDF / JPG



Get involved: Resources

You’re invited to use or join us on any of the following resources:


Publications & Preprints




Media attention

Media mentions of the project or related outputs:


Our team

A great team of open-science JDM Early Career Researcher collaborators joined us to finalize students’ reports for journal submissions:


Completed replications


Note: Those completed replication projects in the table marked as “open” in red are still available for Early Career Researchers to join us and take lead.



Planned replications for academic year 2019-20


Note: Those in the table marked as “open” in red are still available for guided thesis students and interns.




Related JDM mass-replication initiatives

I’ve recently been made aware of similar related initiatives conducting mass replications of JDM findings:

  1. The Hagen Cumulative Science Project
    1. How to Teach Open Science Principles in the Undergraduate Curriculum (preprint)
  2. Replicability and Reducibility of heuristics and biases in Judgment and Decision Making (RR-JDM) @ Linköping University by JEDI lab


Other Registered Report initiatives

Myself with collaborators and students:

  1. Meta-analysis Registered Report: agency constructs and free will beliefs [with Krishna Savani and NTU student team]
  2. Meta-analysis Registered Report: free will beliefs and outcomes [with Krishna Savani and NTU student team]
  3. Meta-analysis Registered Report: values and the dark triad [with Velvetina Lim]


Many-labs type collaborative open-science projects:

  1. Collaborative multi-lab Registered Report: Accelerated CREP – RRR: Turri, Buckwalter, & Blouw (2015) [with Jiaxin Bill Shi and international collaboration] (In Principle Acceptance at AMPPS)
  2. Collaborative multi-lab Registered Report: PSA 006 Moral thinking across the world: Exploring the influence of personal force and intention in moral dilemma judgments
    [with Jiaxin Bill Shi and international collaboration] (reviewed at Nature Human Behaviour)
  3. Collaborative multi-lab Registered Report: STRAEQ-2: Development and Validation of the Social Thermoregulation, Risk Avoidance, and Eating Questionnaire – 2 (Proposal / OSF / OSF Wiki) [with Jiaxin Bill Shi and international collaboration] (in initial stages)




Who is involved?

So far involved in this project:

Big thanks to all involved students and teaching assistants.