The perils of peer review (2024)

By ETIENNE BENSON

gradPSYCH Staff

Competition for grants and publication in top-tier journals is fierce, so it's not surprising that peer reviews can sometimes be highly critical. For most researchers, getting a negative review is not just a rite of passage; it's a way of life.

But, as a student submitting journal articles for the first time, what should you do if you get a truly nasty review-one that clearly crosses the line between a vigorous critique and an unprofessional attack?

Such reviews are rare, but not unheard of. Yale University psychologist Robert J. Sternberg, PhD, APA's 2003 president, has authored hundreds of peer-reviewed publications and served as the editor of several journals, but that didn't stop a reviewer of a journal article Sternberg submitted several years ago from writing what he calls a "savage review."

"[The review] basically said, 'The author of this article really ought to try to work in a field that's more commensurate with his level of mental abilities,'" says Sternberg. "The whole review was at that level, and I've been getting occasional reviews like that throughout my career."

In an effort to keep others from enduring the same, Sternberg has made promoting civility in the review process one of his APA presidential initiatives for 2003.

HOW SAVAGE REVIEWS ARE BORN

Why would someone write an unprofessional review? In many cases it can be hard to tell what motivated a reviewer to heap insults on a study or a researcher. But the following are some factors that Sternberg and University of Arizona psychologist Joel Levin, PhD, APA's chief editorial advisor, say make unprofessional reviews more likely:

• Anonymity-probably the most important factor in enabling unprofessional reviews. "Under the 'cloak of anonymity'-which is characteristic of most, though not all, reviews-the worst, nastiest attributes of reviewers can surface without fear of reprisal," explains Levin.

• Vested interests. Reviewers often have strong opinions about methods and theories in their areas of expertise. Unprofessional reviewers will let those opinions interfere with their ability to provide fair, constructive reviews.

  See Also

  LibGuides: Explore Information: Understanding & Recognizing Peer Review

• Unclear expectations and inadequate training. Beginning reviewers don't always get clear instructions on professional standards for peer review, and those who receive harsh reviews early in their careers may feel that aggressive criticism is expected. Formal training in how to conduct professional reviews is virtually nonexistent.

• Overwork. Reviewers who are swamped with other duties, such as teaching and conducting their own research, may not give a review the time and effort it deserves.

PROFILE OF A SAVAGE REVIEW

The key characteristics of a savage review, say Sternberg and Levin, are:

• Ad hominem attacks that focus on the researcher instead of the research.

• Obvious biases against a particular method or theory that prevent a reviewer from assessing the study on its merits.

• Superficial readings that cause the reviewer to reject a study on the basis of flaws it doesn't really have.

• A tone of voice-arrogant, dismissive or downright cruel-that makes even reasonable criticisms sound like personal attacks.

• Such reviews can cause more than just hurt feelings. They can have serious negative consequences, says Sternberg, including:

• Undermining support for psychological research. If psychologists consistently undervalue each other's work, funding agencies will shift their support to fields whose researchers don't, says Sternberg.

• Giving the field a bad reputation. It is especially hypocritical, says Sternberg, for psychologists-who spend so much of their time trying to help people-to attack their own colleagues.

• Impeding scientific progress. Often unprofessional reviews are not just unpleasant; they're also wrong, says Sternberg. At the very least, they shift attention away from a study's real flaws.

• Discouraging beginning researchers. "I've seen young people in the field get these very savage reviews, and they're devastated," says Sternberg.

Most of the burden of preventing savage reviews lies with reviewers and editors, say APA President Robert J. Sternberg, PhD, and APA Chief Editorial Advisor Joel Levin, PhD, but there are some things you can do as a student author to try to reduce their impact. If you receive what you think is an unprofessional review, Sternberg, Levin and others suggest:

• Getting a second opinion. "Sometimes reviews are just brief, but come across to the recipient as harsh," notes Demarie Jackson, director of APA's journals program. "The author has been patiently waiting for months to hear back and is anxious to get published-careers depend on it-and emotions run high." It can be hard to maintain objectivity, so share the review with someone whose opinion you trust.

• Talking to the editor. Explain in an e-mail or over the phone why you thought the review was unprofessional, and ask for an additional review or set of reviews. Editors are likely to respond favorably to such requests, says Levin, as long as they're phrased politely. "Most editors are sentient beings, too, and have likely been on the receiving end of savage reviews themselves," he explains.

• Making a formal appeal to the publisher. If the editor is unresponsive to your request, contact the publisher of the journal. At APA, the Publications Office, which Levin advises, manages the appeals process.

The most important thing is not to take savage reviews personally, says Sternberg. Almost everyone gets them from time to time, even researchers at the top of their fields, he adds; they say more about the lack of professionalism of the reviewer than they do about the quality of your work.