Highlights and updates from the 9th Peer Review Congress: An insider's perspective
The 9th International Congress on Peer Review and Scientific Publication (Peer Review Congress) began and ended on a thought-provoking note. The three-day hybrid event, held from 8th to 10thSeptember 2022, saw attendance by key stakeholders in the scholarly publishing industry — editors, publishers, researchers, funders, bibliometric experts, librarians, journalists, science communicators, policymakers, ethicists, and anyone interested in scholarly communication.
The 9th Peer Review Congress showcased presentations of research on different practices and experiments with journal publishing workflows, with the aim of creating an evidence base that includes best practices for the dissemination of scientific research. This year, the three-day-long event featured an impressive array of presentations, including 50 plenary session reports, 86 poster session reports, and 39 virtual poster session reports of original research.
Jayashree Rajagopalan, who is the Community Lead at CACTUS and Co-Chair of Peer Review Week 2022, attended the meeting. Here are some highlights and notes from her attendance at the 9th Peer Review Congress.
Day 1 — 8th September 2022
Day 1 of Peer Review Congress 2022 was all about the key issues of research and publication ethics, misconduct, and best practices in research and peer review. There were also riveting exchanges around topics of gender inclusion, pandemic science, the impact of reviewer discussions, and the issue of paper mills. The plenary session reports on day 1 of the Peer Review Congress broadly focused on four main themes:
Authorship, contributorship, and misconduct: The first segment focused on topics including the prevalence of honorary authorship (according to different authorship recommendations and contributor role taxonomy (CRediT) statements), the use of AI-based tools for detecting image duplication prior to manuscript acceptance, and the effect of alerting authors of systematic reviews and that research they cited had been retracted. The session on publication and collaboration anomalies in manuscripts originating from a Russia-based paper mill generated the most conversation and interest among the attendees.
Diversity, equity, and inclusion: Owing to the growing interest around topics of equal representation and equitable opportunities in peer review, this segment drew interest from most attendees. The sessions highlighted studies on women’s representation in peer review of medical journals. They also featured patterns of gender and international diversity of editors and editorial boards (among journals with open access licenses and open science policies). Factors associated with geographical diversity of invited reviewers, and comparison of reporting race and ethnicity in medical journals before and after implementation of reporting guidance were also discussed. Other presentations covered the association between international editorial staff and international publications in leading biomedical journals.
Editorial and peer review models: The sessions covering this theme showcased pilots of journal submission and peer review processes. For instance, one session shared an analysis of submission outcomes and publication timelines for manuscripts submitted to Cell Press’ experimental community review process compared with direct journal submissions. One of the most engaging sessions examined the impact of positive versus negative reviewer-led discussions on the acceptance rates of papers submitted to a machine learning conference. This study aimed to determine the presence of herding behavior in peer reviewer discussions about whether a manuscript should be accepted or rejected.
Pandemic science: This segment highlighted the importance of dissemination of credible research particularly during a global crisis, in light of the COVID-19 pandemic. The presentations covered topics like the epidemiology of scientific output during the COVID-19 pandemic, and a comparison of the characteristics of COVID-19 and non–COVID-19 retractions. Analysis of updates to living systematic reviews related to COVID-19 versus other subjects, a comparison of preprints and peer-reviewed publications of COVID-19 trials, and abstracts of COVID-19-related randomized clinical trials were also in focus.
Day 2 — 9th September 2022
Brian Nosek, Executive Director, Center for Open Science, set the tone for day 2 of Peer Review Congress 2022 with this highly engaging session introducing a new publishing model — Registered Reports — as a step towards reducing publication bias and enhancing scientific rigor. In this model, an initial peer review is done before completion of the research, which can potentially introduce more meticulous checks as the focus of peer review would be on methodology and design rather than the outcome of a study. Brian’s talk emphasizes the need to shift focus from treating publication as the currency of progress to rewarding best practices, to improve the credibility of research findings.
The other overarching themes for day 2 focused on:
Author and peer reviewer guidance and training: The highlight of this segment was the consensus among the attendees about the need and demand for peer reviewer training, and how it can help widen reviewer pools, and improve diversity and inclusion. The session also shed light on the need for mentoring and clear guidelines for reviewers. Original research presented included statistical guidance to authors at top-ranked journals, reminding peer reviewers of the important reporting guidelines to improve completeness in published articles, and an assessment of a structured and mentored peer review curriculum on the quality of peer review.
Peer review: The highlight of this segment was a discussion on the ways to explore, articulate, and show exactly how peer review improved a manuscript. This will give the process much needed visibility and show real time impact. This segment also features interesting research focused on non-biomedical fields. This included the development of a global dataset for peer review in astronomy; comparison of review scores of computer science conference submissions with cited and uncited reviewers; and association between author prominence and peer reviewers’ willingness to review and their evaluations of manuscripts submitted to a finance journal; and studies focusing on manuscript changes between submission/preprint and peer-reviewed journal publication.
Other segments covered studies on dissemination of clinical trial findings, grant review, and funded research. Day 2 also featured poster presentations, which were held in parallel and covered an interesting set of topics.