We have a reproducibility problem. But is it a crisis?

[This is a copy of my blogpost, previously on thespectroscope.com.]

I often bump into biomedical researchers who do not think that there is a problem with reproducibility. They say it’s just an issue of cancer studies or pre-clinical research. As Francis Collins said at a recent conference that I attended, “If you think reproducibility is only a problem in other fields, you need another look at yours.” It is a serious problem, and it is widespread. But is there evidence that it is a crisis?

Most scientists know of the Bayer and Amgen reports showing that far fewer than 50% of published research results are reproducible. But it’s not simply a pre-clinical research plague. After a dozen years as a researcher, I have no problem believing the famous paper from John Ioannidis, Why Most Published Research Findings Are False. If you prefer more empirical and less theoretical studies, you'll enjoy On the reproducibility of science: unique identification of research resources in the biomedical literature by Nicole Vasilevsky​ and colleagues. Vasilevsky's work shows clearly the extent of the problems across different research fields.

I am asking if reproducibility is a crisis because this is crucial for guiding our response to the issue. How much more reproducible was science 20 or 30 years ago? I wish the Bayer study was reproducing another one from them for the 90s. If 70% of published research was reproducible 50 years ago, and now it's the inverse, we have a crisis. However, my suspicion is that we have a problem and reproducibility is a bit worse now than in the past due to the intense hypercompetition in biomedical research, coupled with our 350-year-old publishing system. 

In a few days, I will publish a follow-up post with thoughts on effective approaches to increasing reproducibility. I am assuming that this is an old problem that requires attention, but not a crisis that necessitates panic. If I am wrong, please correct me.

----------------------------

[UPDATE 4/11/15: Just came across Stephen Heard's excellent post from a few days ago, with his answer to exactly this question.]

----

[Comment from Arjun Raj]

I personally think it's an old problem that needs attention, and it's questionable how much attention we should pay to it.  There's just not a lot in the way of sensible ways to enforce reproducibility.  I think that things that work really spread quickly, and things that don't, well, don't.  I think one of the main responsibilities for a PI is to work with their trainees on how to spot when something (a paper, or maybe a whole field) smells a bit funny.