Much concern has been raised in recent years about two related and growing problems with peer review: willing reviewers are hard to find, and when found, review quality is often mediocre at best. A few years ago, the managing editor of a reputable journal remarked to me in an email exchange (regarding the difficulty of finding reviewers): “It is getting worse and worse by the day. Back a few years I sent manuscripts to 2-4 people and I could be sure to have two reports. Today I need to send it to 10 persons to get 1-2 reports. It is awful.”
Recently, as editor for a journal, I was forced to make the following unfortunate recommendation to be sent to the author: “After several invitations, this paper was unable to raise the attention of reviewers; not a single review could be obtained. Because editorial time is limited with many submissions to process, and because the authors are entitled to a timely assessment of their paper, it is in the best interests of the journal to withdraw further consideration of this paper, and also in the best interests of the authors to have an early opportunity to seek an alternative place for publication.”
In some cases of course, an invitation to review may be declined for unavoidable reasons — e.g. because the reviewer is too busy, has a conflict of interest, or does not feel qualified to review the topic or content of the paper. Other personal reasons might be based on concern about avoiding bias in cases where the reviewer has already reviewed the paper for another journal, or based on a preference to remain anonymous, with concern that the review might reveal the identity of the reviewer. In other cases, a reviewer may have none of the above concerns or limitations, but may simply lack sufficient incentive to review — for example because the reviewer finds the topic of the paper uninteresting, and sees no other personal benefit from spending precious time to provide a gratuitous review (especially if it is for a commercial publisher earning big profits).
Some journals have attempted to provide more incentives for reviewers (including for those who are just too busy) by offering things like a free one-year journal subscription, reduced author / article processing fees, more public recognition credit or reputation metrics for reviewing service, and monetary compensation. I am unaware of any evidence
indicating how successful these are, but none of them so far has had any impact in affecting my own willingness to review. Everyone however has a price. [I might be persuaded, for example, by an attractive cash option to work overtime to complete a review that I would otherwise be unable to fit within my regular work schedule].
There is even more to this issue, when we ask the important question the other way around:
Why do people agree to review manuscripts for journals, even when they are already busy?
This is a more interesting question because the fact is: virtually everyone is already overworked. Based on conversations with colleagues, the traditional reasons reported for nevertheless accepting invitations to review are: reviewing service is an obligation to my profession; reviewing helps me to establish a favourable reputation with journal editors; as a reviewer, I can learn how to be a better author/writer, or I can gain early insight into the latest research ideas and discoveries. Some say that when they agree to review (despite being already busy), it is just because they find the topic of the paper interesting.
The extent to which these alleged benefits are realized is difficult to quantify, but they all sound like very fine and noble reasons for making time to review journal manuscripts —
despite that there are 'never enough hours in the day’. However, I am skeptical. I wonder if none of the above reasons, even when added up together, represent the most common reasons for agreeing to review. Could it be that the most common
real reasons are those that people are least likely to admit? Would it not be reasonable to predict that many (most?) people (regardless of how busy they were) would agree to review,
with eager anticipation, every time they are invited to review a paper that cites one’s own work favourably, or disfavourably? — or a paper that is supportive or critical of one's most favourite theory, or least favourite theory? — or a paper authored by a research rival?
Two important reasons then are evident here in accounting for poor quality peer reviewing: One is because the reviewer really doesn't want to review, and one is because the reviewer really,
really does want to review. In the first case, the reviewer very reluctantly agrees
to review — and it shows. This results because the reviewer is already overworked or lacks personal incentive to do a good job, but for some reason feels an obligation to review, and so it is completed with resentment and frustration (often manifesting as an overly critical assessment). In the second case (more common in my view, and much more egregious), a poor quality review results because the reviewer is actually anxious to review, but for the 'wrong' (self-serving) reasons mentioned above. This generates a peer-review bias problem that may be bigger and more systemic than we will ever be capable of detecting. It is the grand un-testable hypothesis—and unspeakable for most researchers—thus ensuring that the peer-review system will forever remain far from perfect.
(
Whatever you do, don't tell the general public.)
