{"id":38070,"date":"2025-09-18T13:00:19","date_gmt":"2025-09-18T07:30:19","guid":{"rendered":"https:\/\/www.editage.com\/insights\/?post_type=video&p=38070"},"modified":"2025-09-18T13:14:33","modified_gmt":"2025-09-18T07:44:33","slug":"ai-does-not-replace-human-peer-reviewers","status":"publish","type":"video","link":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers","title":{"rendered":"AI is Not a Replacement for Human Peer Reviewer with Serge Horbach"},"content":{"rendered":"

AI is only an assistant, not a replacement! If you are someone who tends to expect AI to do the work for you rather than use it as an enhancement tool, you are in for a shock. In this video, <\/span>Dr.<\/span><\/b> Serge Horbach (Assistant Professor, Radboud University)<\/span><\/b> talks about all the good that AI brings to peer review\u2014but also warns against its misuse.<\/span>\u00a0<\/span><\/p>\n

Q:<\/span><\/b> Where do you see the greatest potential for AI to support peer reviewers today?<\/span><\/b>
\nA: <\/span><\/b>Well, I must say that I actually think the\u202fgreatest potential of AI in the editorial\u202fpeer review process is probably not to\u202fsupport peer reviewers currently but much more\u202fto conduct checks that are usually done by the\u202feditorial team. So, I think of checks related to\u202fresearch integrity, image duplication, scanners\u202for reference. Those kinds of checks,\u202fresearch integrity checks, I think that is\u202fsomething where AI can very meaningfully support\u202fcurrently. They might also include checks regarding\u202fthe scope of the journal. So, incoming manuscripts:\u202fDo they align with the journal\u2019s scope? It\u2019s, I\u202fthink, a clear and obvious use case of the current\u202fAI tools to support the editorial process.\u00a0<\/span>\u00a0<\/span><\/p>\n

And\u202fthen when you think of reviewers, I think the main potential currently is really to improve\u202freview reports in the sense of making them, perhaps more constructive or more helpful to the authors,\u202frather than actually writing them from scratch, identifying weaknesses and strengths in\u202fmanuscripts etc. So, it\u2019s more about editing peer review reports to make them more helpful.\u202f<\/span>\u00a0<\/span><\/p>\n

Q: What are the limits of AI when it comes to evaluating manuscripts? Can it handle nuance, ethics, or context?<\/span><\/b>
\nA: <\/span><\/b>I think really the only truly honest answer is that we don\u2019t really know\u202fwhether it can handle\u202fall these aspects in a proper way.\u202fAnd there\u2019s been some research done but \u202f\u202f<\/span>\u00a0<\/span><\/p>\n

It\u2019s really unclear I would say.\u202fThen, the technologies are obviously very quickly developing.\u202fSo, what it might not be able to do now, who knows what\u2019s just around the corner and going\u202fto happen very soon.\u202fBut well, in general, we know that these tools tend to be somewhat more positive than human reviewers. Of course, you can instruct them to be very critical\u202fand give negative reviews, but in general, they tend to come up with fairly positive evaluations,\u202fwhich then, I think again, points to this limit of maybe they\u2019re not so suitable at the\u202fmoment for identifying strengths, weaknesses etc. Maybe at least not to make editorial decisions\u202for recommendations based on that. That is a clear limitation of them currently and instead\u202fthey\u2019re probably better suited to provide feedback or suggestions to improve the\u202fmanuscript rather than to act like this gatekeeping tool or element in the process.\u00a0<\/span>\u00a0<\/span><\/p>\n

And something that we should be very aware of, not necessarily in the sense of it being a limit of AI, but more so that it\u2019s a consequence of using these tools in the editorial or peer review\u202fprocess, is that they come to set quality standards. We\u2019ve seen that very clearly with the text duplication or plagiarism detection software,\u202fwhich then comes to define in the\u202fend what originality really means. What is it that makes us think that this is original new scholarly\u202fwriting? That is when the plagiarism detection software says that duplication is below this and\u202fthat threshold. And the same will happen, if uncritically used, when we implement\u202fAI technologies, who then come to define quality standards in ways that are maybe not always so\u202fobvious. And that\u2019s definitely, well again not an inherent limitation of the technology,\u202fbut a very serious consequence of implementing the technology in systems.<\/span>\u00a0<\/span><\/p>\n

Q: In your experience, what aspects of peer review are uniquely human and irreplaceable?<\/span><\/b>
\nA: <\/span><\/b>Well, we\u2019ve touched on that a bit. I think it\u2019s mainly this aspect of making valued judgments and\u202fdeciding on what really matters and deciding what is really good quality or high-quality research,\u202fidentifying or recognizing the beauty or the excellence of something. I think\u202fthat is something inherently AI technology cannot do. It ultimately, fundamentally\u202fjust works from an input set of criteria or just mimics past standards that have\u202fbeen used in its training data. But independently identifying the core quality\u202fof a piece of work on the review, that I think, remains uniquely human.\u202f<\/span>\u00a0<\/span><\/p>\n

Q: Should peer reviewers be trained to work alongside AI tools? What kind of literacy do they need?<\/span><\/b>
\nA: <\/span><\/b>Well, I think sure if we get to the point\u202fwhere we ask reviewers to use AI tools in their work, then surely they would need training to\u202fdo that properly and the efficacy of using such tools greatly depends on the way in which they\u202fare being used. And that is not self-evident. The output that you would get from these tools\u202fand how to interpret that is, for its own sake, not self-evident. Let\u2019s again take that example of\u202fthe plagiarism detection tools where it seems so obvious maybe from the outside and well you\u202frun the text or the tool on a piece of text and you get a score about how much overlap there is\u202fwith the database against which it was checked. But just that percentage\u2014I think everyone\u202fwho has worked in editorial roles know\u2014just that percentage doesn\u2019t say so much. It really requires you to interpret what that percentage means in a specific context of the manuscript\u202fthat you\u2019ve just run through the system. And the same will apply to using AI technologies for\u202fany task in peer review. And the way in which we should interpret its output requires\u202funderstanding of how the tool works and what its inner workings are. So yes, we\u202fwould have to train reviewers to do that.\u00a0<\/span>\u00a0<\/span><\/p>\n

But again, I say one should ask whether implementing\u202fAI tools as part of the reviewer\u2019s workflow is the optimal place that we can implement these\u202ftools. Currently I think other options would be to make this part of the editorial workflow, the integrity checking for fit with journal scope, etc. Or even on the author side\u202fof the process where an author could run his or her manuscript through an AI tool to get\u202fsuggestions for improvement, etc., and do a bit of a pre-review check on a manuscript to flatten\u202fout any issues that may be flagged by the tool up front. And again, for any actor using the tool in any way that would require a brief or more elaborate training to do\u202fthat properly. And then some have suggested a more elaborate use of AI tools in what\u202fthey\u2019ve called a human-assisted peer review by AI. So, where it\u2019s really turning things around,\u202fnot the AI supporting human reviewers, but rather just keeping the human in a loop to check the\u202foutput of AI tools.\u202fWell, that would surely require some training, but more fundamentally,\u202fI think that is something to be wary of.\u00a0<\/span>\u00a0<\/span><\/p>\n

Clearly these tools hold potential in terms of efficiency, maybe even quality. But we know from research in other areas,\u202fespecially the medical ones, that there might be some issues with these kinds of implementations\u202fwhere it\u2019s humans who check AI output. First, that might not really lead to efficiency gains and\u202fin the end that might still require humans to do the full review and do the actual intellectual\u202flabor required and to review a manuscript or to review AI output regarding that manuscript. But also fundamentally, humans are just not so good apparently at spotting mistakes that these AI\u202ftools might make. Again, that\u2019s been very fairly properly researched in the medical context where,\u202ffor instance, these kinds of AI tools are used\u202fto interpret X-rays or any other scan,\u202fMRI scans or whatever other parts of the human body. A task that was, of course, initially always done by humans. These AI tools tend to be fairly good at doing that. But obviously sometimes\u202fthey make mistakes. And in experiments where humans were then tasked to review the output of an AI tool and spot the mistakes, it turns out that they\u2019re just not so good at that.\u202fAnd they even don\u2019t spot mistakes that they would never make themselves. So, in a control group where\u202fhumans were to interpret the same X-rays or MRI scans etc. and they would make much fewer mistakes\u202fcompared to what the AI tool does and even when the human review of the AI tool is implemented. So, for these kinds of implementations also in editorial peer review I think we should\u202fbe somewhat wary and acknowledge both the limited capabilities of AI tools and humans alike.<\/span>\u00a0<\/span><\/p>\n

Want to know if your paper is ready for peer review? Get your manuscript evaluated by expert reviewers using our <\/span><\/i><\/b>Pre-Submission Peer Review Service<\/span><\/i><\/b><\/a>\u00a0<\/span><\/p>\n","protected":false},"author":8,"featured_media":38071,"template":"","meta":{"_acf_changed":false,"inline_featured_image":false},"new_categories":[],"new_tags":[5830,5869],"series":[5941],"class_list":["post-38070","video","type-video","status-publish","has-post-thumbnail","hentry","new_tags-peer-review","new_tags-peer-review-week","series-peer-review-week-2025"],"acf":[],"yoast_head":"\nAI Does Not Replace Human Peer Reviewers | Editage Insights<\/title>\n<meta name=\"description\" content=\"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI Does Not Replace Human Peer Reviewers | Editage Insights\" \/>\n<meta property=\"og:description\" content=\"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers\" \/>\n<meta property=\"og:site_name\" content=\"Editage Insights\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Editage\" \/>\n<meta property=\"article:modified_time\" content=\"2025-09-18T07:44:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"720\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Editage\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers\",\"url\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers\",\"name\":\"AI Does Not Replace Human Peer Reviewers | Editage Insights\",\"isPartOf\":{\"@id\":\"https:\/\/www.editage.com\/insights\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png\",\"datePublished\":\"2025-09-18T07:30:19+00:00\",\"dateModified\":\"2025-09-18T07:44:33+00:00\",\"description\":\"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage\",\"url\":\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png\",\"contentUrl\":\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png\",\"width\":1280,\"height\":720},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.editage.com\/insights\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI is Not a Replacement for Human Peer Reviewer with Serge Horbach\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.editage.com\/insights\/#website\",\"url\":\"https:\/\/www.editage.com\/insights\/\",\"name\":\"Editage Insights\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.editage.com\/insights\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.editage.com\/insights\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.editage.com\/insights\/#organization\",\"name\":\"Editage Insights\",\"url\":\"https:\/\/www.editage.com\/insights\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.editage.com\/insights\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2024\/09\/editage-insights-logo-1-scaled.webp\",\"contentUrl\":\"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2024\/09\/editage-insights-logo-1-scaled.webp\",\"width\":2560,\"height\":324,\"caption\":\"Editage Insights\"},\"image\":{\"@id\":\"https:\/\/www.editage.com\/insights\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Editage\",\"https:\/\/x.com\/Editage\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI Does Not Replace Human Peer Reviewers | Editage Insights","description":"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers","og_locale":"en_US","og_type":"article","og_title":"AI Does Not Replace Human Peer Reviewers | Editage Insights","og_description":"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.","og_url":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers","og_site_name":"Editage Insights","article_publisher":"https:\/\/www.facebook.com\/Editage","article_modified_time":"2025-09-18T07:44:33+00:00","og_image":[{"width":1280,"height":720,"url":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@Editage","twitter_misc":{"Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers","url":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers","name":"AI Does Not Replace Human Peer Reviewers | Editage Insights","isPartOf":{"@id":"https:\/\/www.editage.com\/insights\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage"},"image":{"@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage"},"thumbnailUrl":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png","datePublished":"2025-09-18T07:30:19+00:00","dateModified":"2025-09-18T07:44:33+00:00","description":"AI is only an assistant, not a replacement! Rather than expecting AI to do the work for you, learn to use it as an enhancement tool.","breadcrumb":{"@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#primaryimage","url":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png","contentUrl":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2025\/09\/Serge-thumbnail.png","width":1280,"height":720},{"@type":"BreadcrumbList","@id":"https:\/\/www.editage.com\/insights\/ai-does-not-replace-human-peer-reviewers#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.editage.com\/insights\/"},{"@type":"ListItem","position":2,"name":"AI is Not a Replacement for Human Peer Reviewer with Serge Horbach"}]},{"@type":"WebSite","@id":"https:\/\/www.editage.com\/insights\/#website","url":"https:\/\/www.editage.com\/insights\/","name":"Editage Insights","description":"","publisher":{"@id":"https:\/\/www.editage.com\/insights\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.editage.com\/insights\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.editage.com\/insights\/#organization","name":"Editage Insights","url":"https:\/\/www.editage.com\/insights\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.editage.com\/insights\/#\/schema\/logo\/image\/","url":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2024\/09\/editage-insights-logo-1-scaled.webp","contentUrl":"https:\/\/www.editage.com\/insights\/wp-content\/uploads\/2024\/09\/editage-insights-logo-1-scaled.webp","width":2560,"height":324,"caption":"Editage Insights"},"image":{"@id":"https:\/\/www.editage.com\/insights\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Editage","https:\/\/x.com\/Editage"]}]}},"_links":{"self":[{"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/video\/38070","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/video"}],"about":[{"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/types\/video"}],"author":[{"embeddable":true,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/users\/8"}],"version-history":[{"count":0,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/video\/38070\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/media\/38071"}],"wp:attachment":[{"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/media?parent=38070"}],"wp:term":[{"taxonomy":"new_categories","embeddable":true,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/new_categories?post=38070"},{"taxonomy":"new_tags","embeddable":true,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/new_tags?post=38070"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/www.editage.com\/insights\/wp-json\/wp\/v2\/series?post=38070"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}