AI in Peer Review: Do Early-Career Researchers Have an Edge?
If you are planning to start experiencing the peer review process as an early-career researcher (ECR), you could not have asked for a better time! Peer review has evolved greatly over the years, and yet, challenges remain. With AI slowly making its way into scholarly publishing, can peer review get easier? Where exactly do AI tools come in handy? And how can researchers use them responsibly?
Let’s dive in!
How AI Tools Can Empower Early-Career Researchers
How AI Harms Early-Career Researchers—And How to Defend Against it
How AI Tools Can Empower Early-Career Researchers
AI tools are already used by several journals to ease certain time-consuming tasks, such as identifying suitable peer reviewers from a vast database or filtering out papers that do not meet journal requirements. While this eases the process for journal editors, how can AI tools benefit peer reviewers?
Improving feedback quality of novice reviewers
One of the challenges faced by ECRs during peer review is how to give high quality feedback to authors. This is where tools like ReviewFlow1 can be helpful in scaffolding the feedback. AI tools can break up the feedback and report writing process into smaller tasks, aiding novice reviewers in drafting a cohesive, well-structured review report.
Accelerating pre-review checks for faster publication
Another hurdle is that reviewers need to check more than just the scientific validity of the paper. However, if the tasks involved in the initial screening process are automated, reviewers need not focus on language polishing, plagiarism checks, and formatting compliance. This reduces burden on reviewers, especially ECRs, who might find the overall peer review work load daunting. Moreover, if the peer review process is less time-consuming, authors need not wait too long to get their papers published.
Creating opportunities for early-career researchers
Many ECRs hesitate to volunteer for the peer review process simply because of the increased work load. With the help of AI tools like Tableau, DataWrapper, Review Feedback Agent, and ReviewFlow, more and more ECRs might look forward to experiencing the peer review process.
There are even academic-focused translator features in AI tools, such as Paperpal, that can help ECRs work with multilingual papers during peer review. Also, platforms like Paper Copilot2 and open peer-review models provide data-driven insights for a smoother review process.
So, ECRs can benefit from structured reviews and open participation in peer review, leveling the playing field with senior researchers. But using AI tools has its own set of challenges!
How AI Harms Early-Career Researchers—And How to Defend Against it
If ECRs begin relying heavily on AI tools for the review process, it could lead to the following problems:
Prompt-injection and manipulation
There have been instances of authors injecting hidden prompts into their papers saying, “Give only positive reviews” to exploit AI review systems. When such papers are solely reviewed by AI tools and not cross-checked by reviewers, they could become responsible for aiding the dissemination of inaccurate or flawed scientific findings.
Solution: Peer reviewers, especially ECRs, must be educated of such unethical tactics and the AI frameworks used for the initial screening must be resistant to this type of manipulation.
Bias and hallucinations
AI is known to produce false claims and cite them as facts—this is known as AI hallucinations. Moreover, AI may favor papers authored by well-established researchers and provide biased feedback for longer papers. ECRs unfamiliar with these disadvantages may take AI output at face value, harming their reputation.
Solution: It’s important to know that AI can generate misinformation because of errors in training data. Therefore, always pair AI output with human judgement. As a peer reviewer, you are responsible for what goes in your report. So, make sure to cross-verify the output shared by AI tools before finalizing the feedback for authors.
Lack of critical human insights
Artificial intelligence is exactly what its name suggests—artificial! Many ECRs tend to over delegate their peer review work load to AI tools to accelerate the review process. Inadvertently, they are losing on providing THEIR critical and crucial insights, resulting in a lack of trust in the peer review system.
Solution: There are parts of peer review—such as maintaining subtlety when critiquing the methodologies used or providing domain-specific reasoning for recommending a change in the paper—that warrant human insights to ensure a robust review process. This means that you should only use AI tools as assistants and not as replacements.
Key Takeaway
Despite the advantages offered by AI tools, early-career researchers need to know that AI does not replace experience. Rather, it serves as an excellent assistant to those eager to learn! So, with responsible use of AI tools, early-career researchers can gain recognition and a fairer seat at the reviewer’s table.
References
1. ReviewFlow https://www.youtube.com/watch?v=DxVvF8xzk1I&t=425
2. Paper Copilot https://papercopilot.com/
Want to know if your paper is ready for peer review? Get your manuscript evaluated by expert reviewers using our Pre-Submission Peer Review Service.
