Pre-registered replications

Quick jump to sections: BackgroundProject summary / Get involved: Resources / Publications & preprints / Media attention / Our team  / Completed replications  / Planned for 2019-20 / Registered Reports


Project pages:



In 2016, following recent developments in psychological science (the so-called “replication / reproducibility crisis”) and gaining my academic independence, I decided to make major changes to my research agenda to prioritize pre-registered replications and pre-registered meta-analyses and focus on the realm of judgment and decision making. The aim was to revisit research findings I once took for granted and re-establish the foundations on which I hope to build my research. I, therefore, decided that all my teaching and mentoring work with guided thesis students will involve either pre-registered replications or pre-registered meta-analyses, atleast as a first step, to examine the classics in the field.

In 2017 I guided 3 masters students at Maastricht University to pre-test this realignment. It far exceeded my expectations. We completed 3 pre-registered replication, 3 pre-registered meta-analyses, and one review paper summarizing the insights gained.

Once joining HKU, in Dec 2017, I decided to scale up and mass-mobilize HKU’s undergraduate students and lead a massive pre-registered replication effort.

In the first two years of running this project, we’ve successfully completed 57 replication projects, most of those with extensions, making this one of the largest replication efforts in social-psychology. For each of the replication projects, we have full pre-registrations, data/code, and all written up in APA style submission ready student reports. In the second semester, most of the replications also included extensions with interesting contributions and insights.

I will continue running this in academic year 2020 with 20+ new replications+extensions. If any of this is of interest to you – lots of ways to join in. I am looking for interested early career researchers to join us, see more info on that below.


I wrote a post for Psychology Today explaining my journey and why I’m doing this project – “Promoting the Credibility Revolution: The Journey to Open-Science“. In March 2020 I gave a recorded talk about open-science and the project (project intro starts at 41:55):



Project summary

WARNING: Preliminary student summarized findings, need to be rechecked and verified.

I summarized findings in a poster in February 2020. Landscape version:

mass pre registered replication project status update portrait poster

Download the poster: PDF / JPG



Get involved: Resources

You’re invited to use or join us on any of the following resources:


Publications & Preprints




Media attention

Media mentions of the project or related outputs:


Our team

A great team of open-science JDM Early Career Researcher collaborators joined us to finalize students’ reports for journal submissions:


Completed replications


Note: Those completed replication projects in the table marked as “open” in red are still available for Early Career Researchers to join us and take lead.



Planned replications for academic year 2019-20


Note: Those in the table marked as “open” in red are still available for guided thesis students and interns.




Related JDM mass-replication initiatives

I’ve recently been made aware of similar related initiatives conducting mass replications of JDM findings:

  1. The Hagen Cumulative Science Project
    1. How to Teach Open Science Principles in the Undergraduate Curriculum (preprint)
  2. Replicability and Reproducibility of heuristics and biases in Judgment and Decision Making (RR-JDM) @ Linköping University by JEDI lab


Other open-science projects




Who is involved?

So far involved in this project:

Big thanks to all involved students and teaching assistants.


How to cite this project?

Please cite as the following:

Collaborative Open-science REsearch (2020). Replications and extensions of classic findings in Judgment and Decision Making. DOI 10.17605/OSF.IO/5Z4A8. Retrieved March 2020 from and