Anyone who has published an article and has gone through a peer review process is a potential reviewer

It’s Peer Review Week, and this is a great time to talk to an industry professional who is deeply involved in peer review and strives to ensure that both peer reviewers and authors have the support they need to make the system function efficiently. I am in conversation with Dr. Bahar Mehmani, who is the Reviewer Experience Lead in the Global Publishing Development department at Elsevier. As part of her role, Dr. Mehmani works on projects that focus on reviewer recognition and aim to acknowledge the contribution peer reviewers make to scientific progress. Dr. Mehmani has also been part of several industry workshops, conferences, and panel discussions around peer review and has written extensively on the topic. She is co-chair of the 2019 Peer Review Week Organizing Committee; a member of New Frontiers of Peer Review (PEERE); and a Council member of the European Association for Science Editors (EASE). Before joining Elsevier, where she has played multiple roles over the years, Dr. Mehmani was a postdoctoral researcher at Max Planck Institute for the Science of Light. She received her PhD in Theoretical Physics from the University of Amsterdam in 2010.

Given her experience as a researcher, author, and publishing professional, Dr. Mehmani is deeply passionate about the communication of research and the need to establish the most efficient systems and resources to do so. In this conversation, Dr. Mehmani tells us more about her work as a Reviewer Experience Lead and talks about the specific challenges peer reviewers face today as well as how they need relevant support to function well. She also shares some interesting views on how the quality of peer review can be improved.

Could you tell us more about your work? What does a Reviewer Experience Lead do?

As Reviewer Experience lead I am helping to improve the researcher experience with Elsevier journals when it comes to the peer review process. I do this by analyzing the peer review process performance at different levels (journal, subject area, business) and finding out what the pain points are and how they can be removed. In this role I work on several projects designed to improve the process and recognize reviewers' contribution to the progress of science, make the process more transparent, collaborative and rewarding.

Based on your interactions with peer reviewers, what are some of the top challenges reviewers face today?

Coincidentally I recently wrote about the same topic reporting my presentation at a panel discussion. I will highlight a few here and defer to the extended list.

• Recognition: The community wants not only journals but also universities and institutions recognize the researcher’s role as peer reviewer. It is important for academics to see funding agencies and evaluation committees taking note of peer review activities as much as they consider the more traditional metrics. 
• Collaboration: Many young researchers perform reviews per their supervisor’s request. Science recently published a post about it. It ties in with lack of recognition, which at least partly can be addressed by journal editors. 
• Bias: Peer review is performed by human beings working in academic hierarchy which is prone to all kind of biases. 
• Transparency: The peer review process is often coined as a black box. Researchers want it to be more transparent, which potentially will make it more reliable.
• Technology: I don’t know of any reviewer who has enjoyed working with old-fashioned and slow submission systems requiring many steps in an inefficient way.

What do you think reviewers expect from authors?

Ideally, high quality research reports. In reality, it depends on the manuscript they review, the time they can dedicate to it, and the circumstances under which they are reviewing. Usually they make it clear to authors by providing their comments to authors. It can be more data points, detailed analysis, different methodology or extra citations! Ultimately, what we sometimes tend to forget is that authors can be reviewers and other way around. It helps if one can put herself in the shoes of authors while preparing the peer review report and make sure the tone and language are set properly and all change requests are focused on improving the quality of the reported research.

What can journals/publishers do to ensure high-quality peer review?

Quality of peer review is not well-defined and, unfortunately, it is not systematically captured. Nonetheless, journals can improve their editorial office processes to make sure that the right type of manuscript lands in the inbox of a reviewer with a matching area of expertise with the least conflict of interest. This can be done by appointing well-established researchers as editorial board members, offering tools and services that enable them to find the best reviewer, and ultimately monitoring the peer review performance of the journal. 

Another important step is to provide reviewers with proper instructions so that they know what the journal expects as a high-quality review report.

Can authors themselves help improve the quality of peer review? How so?

Depending on the journal peer review model and availability of tools for this purpose, authors can provide feedback about their manuscript review reports. In the case of journals that have adopted open peer review this can be done in a more straightforward way as compared to journals that have adopted the double blind peer review system, for example. But at any rate, authors can always inform editors about the quality of their review reports. 

Let’s not forget that one always learns from good examples and remembers it at the time of reviewing a manuscript.

Over the years, there have been several discussions about various types of peer review, each with its own benefits and disadvantages. In your opinion, what kind of peer review works best?

I don’t think there is a universal answer to this question. As I mentioned before we shouldn’t forget that peer review is conducted by human beings performing under different levels of pressure and as such is open to bias. I think the community should decide for themselves and journals should provide them with the tools and services to fulfill their needs.

It is important to study the impact of each model on the peer review process at large scale before deciding which works best. Take transparency for example - our recent study shows that it doesn’t dramatically improve the process.

At what stage do you think early-career researchers should consider becoming peer reviewers?

Anyone who has published an article and has gone through a peer review process, is a potential reviewer.

How open do you think journal editors are to engaging early-career researchers with no prior experience as reviewers?

It depends. Many journal editors I work with welcome the concept. Some journals have programs such as a co-reviewer, encouraging a mentorship program. Many journals are asking members of the community to register as a potential reviewer through their home pages and encourage researchers to get involved in the process. We have a program called Volunpeers enabling journals to systematically engage with researchers interested in being involved in the process.

Any interesting anecdotes about your own peer review experience? Also, do you remember your first peer review as an author?

Reviewer #2!

Yes, and I won’t forget it! I received my first peer review report when I was on the second year of my PhD. It was of the reviewer #2 type and although I got my article published, I was not happy adding reviewers’ suggested extra work, which to this date I don’t think was much of a relevance!

How do you think early-career researchers can get involved in peer review?

First, by publishing and getting first-hand experience of the process. Then, by familiarizing themselves with resources on how to prepare a helpful peer review report. There are many online courses and webpages out there including Elsevier’s how-to-review webpage and Researcher Academy courses.

The theme for this year's Peer Review Week is "quality in peer review," and quality, I believe, is intrinsically linked to the peer review process. I wanted your views on: 

• What does “quality peer review” mean to you? 
• How can we make peer review better? 
• And where do you think peer review is headed?

“Quality in peer review” is a holistic term. It spans from the quality of the editorial process to the quality of peer review report and ultimately to the quality of the published research output.We can make peer review better by engaging with the community, analyzing the performance of the process, learning from other communities and experimenting.

Peer review is headed towards a more inclusive, credible, trustable and user-friendlier future.  It is also becoming a scientific subject matter which can be studied academically.

Thank you, Dr. Mehmani, for an insightful conversation!