Advertisement
Editorial

Expert Failure: Re-evaluating Research Assessment

  • Jonathan A. Eisen,

    Affiliation: University of California Davis, Davis, California, United States of America

    X
  • Catriona J. MacCallum mail,

    cmaccallum@plos.org

    Affiliation: Public Library of Science, Cambridge, United Kingdom

    X
  • Cameron Neylon

    Affiliation: Public Library of Science, Cambridge, United Kingdom

    X
  • Published: October 08, 2013
  • DOI: 10.1371/journal.pbio.1001677

Reader Comments (2)

Post a new comment on this article

Non-anonymous peer review

Posted by Najmanovich on 16 Oct 2013 at 18:52 GMT

Regarding all this latest focus around the issue of peer review, it is my personal hunch that the majority of these problems would be solved if peer review was non-anonymous and open (i.e., published together with the paper) but I have no data to support that. Therefore I'd like to see an experiment done. It would be great to make a 3yr to 5yr long experiment making peer review open & non-anonymous. It would be best if all major publishers joined forces on that. This is a separate issue from open access so most publishers that aren't should not have a problem with this experiment. Unfortunately, it is quite unlikely that Nature, Cell, AAAS, Elsevier, PLOS and other major players join forces. In this case, the next best thing would be a journal with a considerable status, like PLOS Biology to make the experiment with the right to abort it at any time if the perceived quality of submissions decreases.

I think that reviewers wouldn't be as rude, disrespectful and careless about their facts if their names were associated to their reviews and published together with the paper. The often-mentioned reason in favour of anonymity is what I call the David/Goliath issue that is often cited but I believe that it only applies to a small minority of cases. For the most part anonymity only helps to abuse the system. The best way to counter this David/Goliath anonymity argument is that if David's criticism is well-founded and then Goliath punishes David later on, there is a paper trail to prove that Goliath is acting unethically. If David's criticism is nonsense it will become apparent in due course. Everyone should be responsible for their opinions, the moment you add anonymity, you remove this responsibility. Removing anonymity would encourage objectivity, which is what we strive for in science. This would also serve as a public acknowledgement of the work of reviewers and the quality of their reviews and could more clearly serve as one more tool to help evaluate scientists as this an important part of a scientist's job. This is why an important magazine such as PLOS Biology is necessary for the experiment, to attract good research and good reviewers who will benefit from being publicly associated to a leading journal. Once this experiment is concluded, we can judge what impact this change made on a number of parameters (reproducibility, retractions, citations, altimetrics, etc.) based on comparisons with previous years of PLOS Biology.

The major problem with mandatory non-anonymous open peer review is that it could and probably would at least at first, make it more difficult to find referees as people would be afraid to risk associating their names to their opinions... but there are workarounds that problem. For example, the associate editor could and probably should play a more active role in refereeing the reviews before a review is made public and sent to authors. I myself as AE in PLOS One already do that a lot, filtering out a lot of nonsense that although sent to the author, I as AE don't request the author to act upon. An Academic Editor with stronger role working in the direction of the reviewer in addition to that of the author would serve as a security net for reviewers assuring them that their comments are pertinent. This would overall increase the quality of the entire publishing process.

There is at least one journal that already implements non-anonymous peer review (F1000Research, of which I am also an AE) but to have the proper controls in the experiment proposed above, it is necessary to use an already well-stablished journal such as PLOS Biology. I am also aware that currently at least PLOS One gives referees the choice to make their names known to the authors but the experiment will only work if done consistently.

PLOS has been a leader in changing the scientific publishing environment and it is ripe time for PLOS to take the lead and improve the scientific method of publishing and thus increase the quality of science overall. This experiment, in my view, not only will be successful but it will also further raise the quality of the manuscripts published by PLOS.

No competing interests declared.