Peer review as a system: Can we do better?
I am currently having a wee bit of a spat in the literature regarding a published paper in a field within my specialism (electron backscatter diffraction). The process of trying to resolve this in a reasonable fashion has been a bit messy, and very time consuming. Here, I share thoughts on how we can improve scientific publishing as a whole.
1. Journals are useful
I feel that journals add value. There is an ever increasing amount of science being reported, of a range of qualities, and having to submit your work to a journal, to be subjected to peer review, and contributing to the established literature builds an ethos of getting the work to be in a reasonable state for wider dissemination, discussion, and adding value to the bank of scientific knowledge.
Journals and their publishing houses can also offer a stable, well-resourced, and experienced avenue for authoritative scientific communication. These come in a variety of flavours, from those within massive “for-profit” groups, or within a variety of newer open access flavours.
Discussions of the benefits of different publishing methods within a journal framework is scope for another day. Yet, the adding value to a process inherently has a cost, and we must rationalise how to make these costs work for the benefit of scientists and our funders.
2. Pre-prints can make a valuable contribution
Moving purely to a pre-print culture enables science to be disseminated more quickly and it opens up the scientific community. Furthermore, the pre-print architecture can move science from beyond pay-walls, across cultures, and remove the constraint of scientific interaction from pricey conferences or membership to elite institutions.
At the moment, I do not think that pre-print systems should replace journal structures. I feel that the formalisation of journal publications is useful; as I do not believe that, even in a world of post-publication peer review, there will be enough substantive and invested checking of rigour and quality of work without the creation of manuscripts (they are also one of the best technology transfer systems, often facilitated by widgets, data, code or protocols).
Furthermore, I feel that if we move towards growing our use and reach of a pre-print structure, stewarded by collections of scientists and engineers, we can enhance and grow the current offering.
3. Pre-publication peer review is useful
Pre-publication peer review provides an independent (at least) once over of a manuscript and can provide glaring holes in the argument to be explored between experts.
It is not flawless, as we are not experts in everything and there are vested interests (direct £ conflicts, conflict of reputation, self-investment, ego, and ego-on-behalf of the journal).
In an ideal world, all parties in the publication process would have adequate reward (both £ and reputation) for being expert reviewers and lending valuable time to the process, and that argument in correspondence would trump ego and personal reputation, but we are only human and the system is long-standing.
Post publication peer review is an interesting development, but I am time-poor and there is not a culture of reward or incentive to support its long-standing growth at present (and we risk individuals gaming the system even more than is presently afforded with editorial processes).
4. Scientists need better training and exposure to managing conflicts in the scientific literature
I have been involved in two academic disagreements in my career so far; both feel unrewarding on a personal front (time, £, energy, and outcomes), and both have been costly (time, £, energy and reputation). Each time I go through this process, I learn more about how other people react and how conflicts can be managed (mostly in hindsight).
It feels like there is space to explore these issues more widely, perhaps through blogs and social media to start, but maybe also with case studies. As an idea for my group, we have recently started a Journal Club, to explore different writing styles, learn new techniques and ideas, and to track how different cultures report on their scientific findings. I think I am going to find a good comment/critique for us to explore and discuss together, it’s likely to be fascinating.
Moving forward
How can we open up the scientific dialogue and enable science to push us forward?
I feel that we need to open up the system.
Let’s publish more peer review comments, let’s make open data sharing a requirement, let’s encourage and support authors to submit pre-prints.
We can still maintain a “gold standard” of scientific publication, but provide more open checks and balances along the way. Furthermore, in opening up our processes, we can learn together and work to improve the system and improve everyone's expertise.
Opening up the system should improve everyone’s communication and re-balance the system towards its intended objective — getting the best ideas out there to the widest audiences, testing important hypothesis, expanding our understanding & enabling the best decisions to be made for society.
Dr. Ben Britton is a Senior Lecturer and Royal Academy of Engineering Research Fellow at Imperial College in London. This article was originally published on Dr. Brittion's Medium blog (available here) and has been republished here with his permission.
Comments
You're looking to give wings to your academic career and publication journey. We like that!
Why don't we give you complete access! Create a free account and get unlimited access to all resources & a vibrant researcher community.
Subscribe to Journal Submission & Peer Review
