Enhancing the grant peer review system: Overview of Publons global survey

Peer Review Week 2019 may be over, but peer review continues to be in focus. After Sense about Science shared the results of its peer review survey last month (which looked at peer review from the lenses of trust and quality), Publons this month released the results of its grant peer review survey. Titled Grant Review in Focus, the survey sought to gain vital contributor perspectives on the grant review system. The survey is based on interviews with about 4,700 peer reviewers registered with Publons, with the majority based in Europe. Additionally, the survey sought insights from several leading research funders that the peer reviewers have collaborated with.

The grant peer review system is important for all stakeholders. It helps funders arrive at a judicious decision, thus enabling optimized funding. This is especially important considering the current scenario where several countries are imposing cuts on research funds. For researchers, gaining grants is especially critical as grants are treated as a metric for performance evaluation and career progression. On the other hand, for reviewers, apart from providing an opportunity to “give back to science,” grant peer reviews help ensure that the right science is getting funded.

Keeping this in mind, let us have a look at the key results from the survey.

• Researchers overall believe in the system. A majority (78%) agreed that it is the best prevailing method for ensuring the right research is funded. In the words of one of the (anonymous) respondents, “I have found it [the system] to be rewarding, valuable, and important.”
• The grant process is extremely competitive, with only 10-25% applications having received funding. Additionally, this rate is declining, reflecting the global trend of reduced spending on research.
• As is the case with the manuscript review process, the grant review process is time-intensive. Funders spent between two and six hours on average looking for grant peer reviewers. The average number of reviewers invited to secure a confirmation was three, that is, two-thirds of the effort was spent without result. On the review side, reviewers spent about 10 days of the year reviewing grant applications.
• Over half (56%) of the respondents reported being dissatisfied with the recognition they receive. As one respondent complained, “The thank you mail [the most common method of acknowledgment] is typically computer-generated… the granting organization could not care less.”
• Community contribution is a big reason for doing grant reviews. While the primary reason for doing grant reviews is that it is a part of the job of researchers, a close second was the need to give back to the field.

The survey re-affirms some of the benefits of the grant review system and provides insights on addressing some of the grievances with the system. However, it is important to note that while the pool of researchers interviewed was sizeable, it was a restricted pool, limited to Publons users. Thus, there may be some bias in the results. As Anna Severin, a sociologist at the University of Bern, Switzerland studying peer review, pointed out, a “Western bias” is perceptible among the respondents: about 50% were from Europe and only 3% from China.

Publons has acknowledged that the results may not be representative of scientists across the world. Nevertheless, the survey adds significantly to our understanding of and growing conversations around the peer review system.

Related reading:

• Top tips for researchers on how to avoid grant proposal rejection
• 9 Core parts every grant proposal must contain
• Peer reviewers tell all: A Q&A with expert reviewers

References:

• Grant Review in Focus [Global State of Peer Review Series]
• Thousands of grant peer reviewers share concerns in global survey