 |
|
You are currently at :: Resources > Articles on Research Publishing |
| |
 |
 |
| |
Peer Review: The Process and Its Importance |
| |
|
| |
Introduction
Peer review is a process widely used by scientific journals and funding agencies to scrutinize the manuscripts and grant applications submitted to them for publication or funding. Peer review is usually conducted by experts in the field who have been asked to evaluate the submitted manuscript/grant application for originality, scientific merit, and importance of the research work. They also have to decide whether the manuscript/grant application fulfills the journal/funding agency's criteria for acceptance.
What is peer review?
Peer review (also known as refereeing) is the process by which the work done by a researcher is evaluated by one or more of his/her peers (i.e., researchers in the same field). The organization Sense about Science has defined scientific peer review as The evaluation of scientific research findings or proposals for competence, significance, and originality, by qualified experts who research and submit work for publication in the same field (peers).
Read more at http://www.senseaboutscience.org.uk/index.php/site/project/33.
What is peer review used for?
The process of peer review is mainly used by
|
| 1. |
Scientific journals for the assessment of submitted manuscripts.
When a journal receives a manuscript, it needs to determine whether the work presented is suitable for publication. The editor of the journal, therefore, sends the submitted manuscript to 2 or 3 experts in the same field and asks them to assess it for scientific merit and credibility of content, quality of writing, and suitability for publication. This process is also termed refereeing. |
| 2. |
Funding agencies to assess the scientific merit of grant applications.
Grant applications include a description of the specific research protocol for which the funds are being requested. In order to assess the significance of the research, the scientific merits of the research design, and the relevance of the proposed research to the aims of the funding agency, these applications are scrutinized by a panel of subject experts. |
| 3. |
University departments or government agencies to determine tenure/ provide grants.
Another area where the process of peer review is commonly used is in the assessment for tenure of faculty members. Experts in the field, very often from another university, are invited to form a review board which then considers the research work, publications, etc., of the scientist applying for tenure. Funding agencies may also use a similar process to decide whether to fund the research studies in a particular department or university.
|
|
 |
How is peer review conducted?
The process of peer review actually starts when the manuscript or grant application is submitted for consideration. The journal editor (usually a senior academic in the same or related field, i.e., peer) scrutinizes the manuscript to decide whether it meets the journal's criteria for publication. Once it is found suitable, the manuscript is sent to 2 or 3 subject experts (who have previously agreed to act as peer reviewers for the journal) or to the review committee of the funding agency. The process of peer review is depicted below in a flow diagram:
|
|
Some journals have very stringent acceptance criteria: only 40% of articles submitted to Nature make it to peer review, and of these, less than a quarter reach publication. Only 7% of articles submitted to Nature actually get published!
( http://www.nature.com/nature/peerreview/debate/nature05535.html)
Types of peer review
Traditionally, the process of peer review is carried out without the authors being aware of the reviewers' identities. The reviewers, on the other hand, are aware of the authors' identities. This is referred to as a blinded or masked review. The advantage of this type of review is that the anonymity provided to the reviewers encourages them to critique the submitted manuscript/grant application fearlessly.
To make the review process appear more impartial, many journals and funding agencies today submit manuscripts/grant applications to a process of peer review where both authors and reviewers are unaware of each others' identities. This is referred to as the double-blinded or double-masked review. The advantage of this form of peer review is that it appears to be a more impartial and unbiased form of peer review.
A new type of peer review process is the open peer review in which authors and reviewers are aware of each others identities. Online, open access peer review is being tried by a number of journals and websites, given the increasing trend of online publications. One example of a successful online site is arXiv—an electronic repository started in 1991 for pre-prints of papers in Physics, Mathematics, and related disciplines (http://arXiv.org). Researchers deposit their as yet unpublished papers (termed e-prints) in this repository and viewers can make their comments on the work. The researchers can then use the comments to make changes in their work prior to submitting the manuscript for publication.
A number of online journals such as Philica (http://www.philica.com), the Journal of Interactive Media in Education (http://www-jime.open.ac.uk/), and Biology Direct (http://www.biology-direct.com/) have also tried to introduce, in varying forms, the concept of online peer review. However, many reviewers and authors appear reluctant to participate in open peer review processes, as was found by the journal Nature when it conducted an experiment in which authors were invited to permit their submitted manuscripts to be reviewed online. Only 5% of authors agreed to participate in the experiment. Interestingly, only 54% of these articles received reviews online. The chief reason may be a reviewer's desire to remain anonymous when making critical comments about a paper.
Read more at http://www.nature.com/nature/peerreview/debate/nature05535.html.
Why is the process of peer review so important?
Why is the process of peer review given so much importance despite the drawbacks faced by the system (discussed in more detail later in this article)? Most scientists favor the use of the peer review system for the assessment of research papers and grant applications and are reluctant to end its use due to various reasons such as
|
| 1. |
The reviewers' comments on the research proposal in a grant application or on the work presented in the submitted manuscript contribute to improving the quality of the proposed research or manuscript. |
| 2. |
By participating (as reviewers) in the process of peer review, senior scientists can help to maintain and set standards of quality for research work in a particular field. |
| 3. |
Unlike open access review, the author is assured that, in the traditional peer review process, the reviewers' comments have been made by those best qualified to comment on the work. |
| 4. |
An article which is published after having undergone a process of peer review can be considered as having attained a certain minimum standard in (i) the research it presents and (ii) its contribution to scientific knowledge. |
| 5. |
A combination of the process of peer review and journal Impact Factor ratings results in the best research being published in the best journals and so on. Peer reviewers therefore save the reader a lot of time and effort by helping to sort the published literature into an order of descending quality.
|
|
|
What are the demerits of the system of peer review?
There are some demerits to the peer review process, which are often referred to by the critics of this system. These are as follows:
|
| 1. |
Selection by the journal editor
The process of peer review begins in the office of the journal editor, who has to decide at the outset whether the submitted manuscript should be considered for inclusion in the journal. There is a possibility of the editor being influenced at this stage by personal bias. Once the manuscript passes this hurdle, the next hurdle is the selection of the peer reviewers by the journal editor. The editor may choose to send the manuscript to a harsh reviewer, knowing that the chances of it being turned down for publication are high.
|
| 2. |
Possibility of reviewer bias
Peer reviewers are scientists working in the same field as the author or grant applicant. They may subscribe to a school of thought biased against the basic premise of the research work submitted, leading to rejection of the manuscript/application. It has been found, as fields of research get more specialized, that it is often possible for the reviewers to identify the authors of the manuscript/grant application and there is a fear that they may allow their personal bias against the author or grant applicant to influence their recommendation to the journal/funding agency.
|
| 3. |
Misappropriation of research
Some reviewers who are working on a research project similar to the one they are reviewing may delay the publication of a paper until their own (similar) paper has been published. Unscrupulous reviewers may reject the manuscript or grant application only to submit an almost identical manuscript/grant application themselves.
|
| 4. |
Inability to detect flaws
Peers can make mistakes while reviewing the submitted work. As you will see in the examples given below, there are times peer review has been unable to detect the flaws in the research or whether the work is genuine or fraudulent. However, these lapses happen infrequently and should not be the reason for discontinuing the process of peer review.
|
| 5. |
Misuse of the system by authors
Some journals have a policy of not inviting persons thanked in the Acknowledgements section of a paper to be peer reviewers for that paper. Therefore, an unscrupulous author who does not want certain scientists to review the work submitted (for fear of a biased or harsh review), may deliberately mention them in the Acknowledgements section. Authors may also present a large study as several small papers instead of one comprehensive paper (the salami slicing school of publication!), in an attempt to increase the list of papers in their resume.
|
| 6. |
Scientific standards can be met even without peer review
One of the arguments against the process of peer review is that there have been research papers published which have not undergone peer review, but have still met the highest standards of scientific research and made seminal contributions to our understanding of a particular subject (see examples below). There are, however, not many papers which fall into this category.
|
|
As part of their excellent presentation titled Peer review in the Google Age, Dominy and Bhatt list some famous papers where the serious issues mentioned above interfered with the process of peer review. One of their slides is adapted and presented below.
The complete presentation can be seen at
http://eprints.rclis.org/archive/00006004/01/Peer_Review_in_the_Google_Age%5B1%5D.ppt
|
|
| Famous papers that were published and did NOT get peer reviewed: |
 |
Watson & Crick's 1951 paper on the structure of DNA in Nature.
|
|
Abdus Salam's paper "Weak and electromagnetic interactions" (1968). Led to the Nobel Prize.
|
|
Alan Sokal's "Transgressing the Boundaries..." in 1996 turned out to be a hoax. Now known as the Sokal Affair.
|
| Famous papers that were published and passed peer review that later proved to be fraudulent: |
|
Jan Hendrik Schon (Bell Labs) submitted and passed through peer review 15 papers, published in Science and Nature (1998-2001), found later to be fraudulent.
|
|
Igor and Grichka Bogdanov 1999 & 2002 published papers in theoretical physics believed by many to be jargon-rich nonsense.
|
| Famous papers that got rejected that later turned out to be seminal works: |
|
Krebs & Johnson's 1937 paper on the role of citric acid on metabolism was rejected by Nature as being of "insufficient importance," was eventually published in the Dutch journal Enzymologia. This discovery, now known as the Krebs Cycle, was recognized with a Nobel prize in 1953.
|
|
Black & Scholes 1973 paper on "the pricing of options and corporate liabilities," rejected many times, was eventually published at the intercession of Merton Miller in the Journal of Political Economy. This work led to the Nobel Prize.
|
|
Conclusion
The process of peer review is an old one and provides a valuable service to the research community. While alternative methods of assessment have been tried, such as open access review, they have not been very successful. Subjecting manuscripts and grant applications for peer review helps to maintain, and set, standards of quality for research work in a particular field. A combination of the process of peer review and journal Impact Factor ratings results in the best research being published in the best journals, providing readers with a rough grading scale as to the quality of the work in the paper they are reading. While the system has certain defects, as seen in the examples above, it is still the most widely used and accepted form of rating research work. Other forms of review, such as open access review, are still being tried out and the use of peer review for the assessment of scientific merit will continue until a better alternative is established. |
| |
|
|
|
