The year 2011 introduced first signs that something is very wrong about the way we’ve been doing science, but it took till about 2016 with failures of mass replication projects for us to begin to understand just how bad things are.
I have since committed to trying to better understand the situation (coined “replication/reproducibility crisis”) and to try and promote solutions to overcome inherent problems (coined “credibility revolution”).
In this page is a collection of easy-to-understand videos summarizing the main issues, and that appears below some of the open-science initiatives and education that I am personally involved in.
My Journey to Open Science
I wrote a post for Psychology Today explaining my journey and why I’m doing this project – “Promoting the Credibility Revolution: The Journey to Open-Science“.
In the 2020 autumn courses I gave the following opening lecture on the science crisis :
If you prefer non-course seminar talk style instead, then in March 2020 I gave a similar recorded talk about open-science (slides) which also includes some Q&A at the end:
If you prefer a shorter 20 minutes version, then in April 2020 I took part in the following debate with the Brazilian replication initiative, we each talk for 20 minutes and then have a debate for an hour (my slides):
In 2019 the students in my PSYC2020 Fundamentals of Social Psychology course joined me in writing the first book covering the science crisis: “Taking stock of the credibility revolution: Scientific reform 2011-now“. We hope to keep improving on this in the future (though there are now more professional books, see list below), but for now [download link]:Taking stock of the credibility revolution: Scientific reform 2011-now
Open science education and initiatives
Yet issues go far deeper, and to learn more you can see some of the following resources I developed over time:
Open-science initiatives that I’m involved in or track closely:
For more initiatives, see PSA Meta-research hub
Replication / reproducibility crisis
I support the Open Science movement, and recommend any scholar in any discipline to educate themselves about the advances in science, problems and solutions. Here are some quick simplified videos to get you up to date on the problems as of 2016. I try to keep track of articles on my WIKI. You are welcome to read more about my related work on pre-registered replications and meta-analyses.
Issues with stats
To better understand how we test things in science (and psychology) using statistics, PBS has a good intro “Prediction by the Numbers” that discusses statistics, what Null Hypothesis Significance Testing (NHST) p-values are (some term as the “significance threshold”), possible biases in inferences, and the associated issues with NHST and p-values that have contributed to the current ongoing crisis.
Credibility Revolution: Moving forward
For a more scientific in-depth lecture, Brian Nosek of the Center of Open Science does a great job with this talk:
And, no, it’s not just psychology. Here’s from Prof. John P.A. Ioannidis in 2019 on “Scientific evidence: reproducible and useful”:
Promoting open-access science
This movie is a good summary of the issue – Paywall: The Business of Scholarship
Books: Psychology as a Science (and the replication crisis)
- Science Fictions, by Stuart J. Ritchie
- The Seven Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice, by Chris Chambers
- Calling Bullshit: The Art of Skepticism in a Data-Driven World, by Carl T. Bergstrom and Jevin D. West
- Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference, by Zoltan Dienes
Podcasts on open-science, academia, & credibility revolution
Podcast episodes on open-science and replication crisis