What fosters trust in peer review: Perspectives of Chinese researchers
Trust is the bedrock of peer review. Scholarly progress is sustained largely by an institution relying on professionals and volunteer experts to do their job with integrity. The theme for this year’s Peer Review Week—trust in peer review—is timely as well as apt. The scholarly publishing community has been trying multiple approaches to strengthen peer review in the past few years, and the current pandemic has been putting a strain on peer review as never before.
How much do researchers trust peer review? Which factors influence their trust in peer review? How has their trust in peer review weathered the COVID-19 crisis?
In the weeks leading to Peer Review Week 2020, we asked these questions to researchers in China—the leading research producer in the world and one of the countries that has produced the most research related to COVID-19. This post summarizes the results of a survey conducted by Editage.
Survey period: August 20 to September 5, 2020
Total number of respondents: 545
Main questionnaire components: One question asking respondents to rate level of agreement with statements on a scale of 1 to 5 [strongly agree, agree, neither agree nor disagree, disagree, strongly disagree], one question asking them to rank trust-related factors in the order of importance, and one question about effect of COVID-19 on their trust in peer review
Survey dissemination channel: WeChat
Demographic distribution of respondents
The majority of respondents were male, Ph.D. degree holders, and early-career researchers. Most had published over 5 papers, and the largest proportion belonged to the fields of physical sciences, engineering, and related subject areas.
Around 59% respondents said that they found peer reviewer comments useful, and an overwhelming majority—79%—said that they trust the quality of papers that have undergone peer review more than the quality of those that have not.
(Note: In the figures below, overall agree and overall disagree refer to the total numbers of respondents who chose strongly agree/agree and strongly disagree/disagree, respectively.)
A large majority of the respondents agreed that perception of reviewer bias of any type influences how much they trust peer review and that their trust in peer review would improve if global standards for peer review were established.
We asked participants to rank seven factors based on how important each would be in establishing their trust in a journal’s peer review process. The responses were analyzed by assigning mean ranks to each factor.
The top three most important factors were whether a journal has had a record of retractions, whether it reveals reviewer names post-review, and whether it offers rewards/remuneration to peer reviewers.
Finally, we asked participants how their confidence in peer review was affected by the ongoing COVID-19 situation. The majority—60%—said that it had remained the same as before, and 11%, in fact, believed that it had increased.
1. Value of peer review. While researchers may be slightly divided about how valuable they find peer review for their own manuscripts, the majority tend to place great importance on peer review in general as a quality-check process in scholarly communication.
2. Fairness and bias. Efforts to improve fairness in peer review by minimizing potential reviewer bias are important. This is underscored by the large majority of respondents saying that perceived bias affects their trust in peer review and by the fact that author anonymity and disclosure of reviewer identity ranked as more important trust-related factors than the transparency of overall review processes and peer reviewer selection policies. Previous surveys have consistently shown that researchers tend to prefer double-blind peer review to other review types. The main reason for this is that the double-blind format is believed to enforce greater objectivity. Approaches to improving reviewer objectivity and accountability are important to establishing trust in peer review, especially among authors from the Global South.
3. Retractions and reputation. Authors care deeply about whether journals have a history of retractions/expressions of concern, since this ranked at the top of factors that influencing trust in a journal’s peer review process. While the survey did not ask respondents to justify the ranks assigned, this result may be interpreted in two ways: (1) authors may believe that retractions or expressions of concern reflect poorly on journals’ peer review processes, or (2) they may believe that these are healthy signs that journals are willing to take action and correct any lapses in peer review.
4. Global peer review standards. Establishing global standards can improve not only the quality of but the trust in peer review. It is understandable that peer review processes are determined by many journal-specific factors, and, therefore, peer review quality can vary widely across journals. Nevertheless, perhaps it is time to consider applying standards to at least some aspects of peer review, such as reviewer training programs and expected review outputs. Peer review was already found to be among the top three areas of academic publishing that researchers worldwide want changes in, especially with respect to quality and processes. And defining standards may be even more relevant for urgent, high-impact research, such as research on COVID-19.
5. COVID-19 and trust. COVID-19–related pressures may not have extensively damaged trust in peer review among researchers, even though concerns are being voiced about how peer review quality is being maintained and the actual impact of these pressures on peer review is not easy to gauge in the short term.
This article is powered by , a platform that combines AI-harvested data with human curation to help researchers working on COVID-19–related topics find the literature and insights they are looking for.
For further insights related to trust in peer review during the pandemic, join us in with experts on Sep 21, 2020. This webinar is part of the organized by Cactus Communications.
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.