Download free handbook

You are here

Is the reproducibility crisis real? An overview of Nature's survey

Sneha Kulkarni | Jun 3, 2016 | 8,341 views
Is the reproducibility crisis real? An overview of Nature’s survey

Irreproducibility of published results is one of the most discussed topics within scholarly circles and is often referred to as a modern-day crisis that science is facing. The failure to reproduce results is concerning because it is indicative of misjudgment, poor experimental design, selective reporting, or a statistical fluke. However, the lack of a commonly agreed definition of reproducibility, and the various opinions on what underlies the issue and how it can be resolved prompted Nature to conduct a survey.

The survey conducted by Nature quizzed about 1500 scientists about their attitudes towards reproducibility and how results can be made more reproducible. Some of the most significant survey results are listed below:

1. Is the reproducibility crisis a reality?

Of the participants who took the survey, 52% agreed to the view that “there is a significant 'crisis' of reproducibility,” whereas only 7% stated that they did not think reproducibility is a problem.

2. Are any fields free of this problem?

When questioned about reproducibility in their respective fields, 73% believed that at least half of the papers in their field are reliable, revealing contradictory attitudes. Of them, physicists and chemists were most confident about the quality of research published in their disciplines.

3. How challenging is it to publish replication studies?

24% of the respondents reported that they published a successful replication whereas 13% published a failed replication. However, several scientists involved in the survey stated that they found it challenging to get a failed replication published.

4. How often were the participants contacted with regard to their published results?

Remarkably, less than 20% of the participants reported that they had been contacted by another researcher to discuss the irreproducibility of their work.   

5. What are the factors that lead to irreproducibility?

Selective reporting and pressure to publish were considered the most responsible factors underlying the reproducibility problem by more than 60% of the respondents. Another reason cited by around 50% of the participants was “low statistical power or poor analysis.”

6. What factors could augment reproducibility?

According to 90% of the respondents, “more robust experimental design,” “better statistics,” and “better mentorship” are the three factors that can ensure reproducibility. “Journals enforcing standards” was rated the lowest with 69% (which is quite high) of respondents voting for it.     

The responses of the scientists make it evident that the reproducibility crisis is indeed a reality. While as many as 70% of the surveyed scientists reported their failure at replicating another researcher’s results, only a few were contacted by another researcher to report failed replication. This clearly shows a communication gap among scientists. Moreover, the hesitation of journals to publish failed replications is another indication of a loophole in the publishing system that has a direct impact on the pace of scientific progress.

The roots of this problem run deep, and all the stakeholders of science should work together to create a more transparent and robust system. While funders and institutions are working towards developing policies that improve reproducibility, every researcher should take it as a responsibility to ensure that their findings are free of any flaws and are reported without any omissions. If scientific progress is summed up as looking further by standing on the shoulders of giants, we need to ensure that every giant stands tall and firm. 

Recommended reading:

Irreproducibility: The soft underbelly of science

Why are replication studies so rarely published?

How publishing replication studies and negative results helps science

Republish

Like this article? Republish it!
Knowledge should be open to all. We encourage our viewers to republish articles, online or in print. Our Creative Commons license allows you to do so for free. We only ask you to follow a few simple guidelines:
  • Attribution: Remember to attribute our authors. They spend a lot of time and effort in creating this content for you.
  • Editage Insights: Include an attribution to Editage Insights as the original source.
  • Consider a teaser: Yes, that’s what we call it…a teaser. You could include a few lines of this post and say “Read the whole article on Editage Insights”. Don’t forget to add the link to the article.
  • Re-using images: Re-publishing some of the images from our articles may need prior permission from or credit to the original image source.
  • Quick and easy embed code: The simplest way to share this article on your webpage would be to embed the code below.

 

Please copy the above code and embed it onto your website to republish.
Join a community of 179000+ researchers
Editage Insights offers a wealth of free resources on academic research and publishing. Sign up and get complete access to a vibrant global community of researchers. Gain expertise & share your own with authors and others involved in scholarly publishing.
By clicking 'Join Now', you agree to our Terms & Privacy Policy.
Having trouble registering/logging in? Contact us
Q & A

Have your own question?

Related Categories