Tips for Writing Article Reviews 

By Nick Wagner

Disclaimer: The following recommendations reflect my training and experience alone. This is how I do it. It’s not the only way to do it.

General Notes

• Review the research, not the researchers. Refer to the manuscript (e.g., “the introduction of the manuscript would be improved by the inclusion of X literature”) rather than the authors (e.g., “the authors failed to include X in the introduction”). Be nice!

• Be specific. Your task is to assess the specific research that has been submitted for publication. This is not to suggest that you shouldn’t comment on how the manuscript fits with the literature as a whole, but this assessment should provide specific examples or explanation. If you don’t think the manuscript will make a contribution to the literature, say so and say why, specifically.

• Relate everything to the study goals presented by the authors. They’re out to achieve their goals, not yours, so their work should be assessed in the context of their aims.

• Focus (almost entirely) on threats to validity. Your review should focus solely on potential threats to validity, both external and internal. Are there plausible alternatives to the inferences that are being drawn? Are there confounding influences which have not been considered? Are there barriers to generalizing study inferences to the populations of interest? Do the conducted analyses align with the stated goals of the study, and are they appropriate for the types of data being used?

• Ask questions. This can be a dialogue. Do not hesitate to ask specific questions of the authors, so long as the answers to these questions serve to improve the paper. If the reliability of a measure is not described or a procedure is not clear, ask. But also suggest this information be included in the revision as it will be important for all readers. If the answer to your question isn’t relevant to the eventual readership of the article, think hard about whether it needs to be asked.

Writing the Review

• Schedule one hour to write the review and do your best not to go over.

Reviewing an article will likely take more time early in your career, but I’ve found that putting a limit how long I’m willing to spend on the review helps to focus my work without negatively impacting the quality of the review.

• Start the review by stating your view of the main goals of the article.

The first paragraph of your review should state what you think the article is about. This shows the authors that you read it. They know they have some work to do if they read the review and see that your opening paragraph misses what they had intended to be the main point(s) of their paper. This paragraph should also provide your global assessment of the paper.  This is usually how I open:

“The article entitled “xxx” examines xxxx. [insert statement about overall view of how successfully addressing these goals will contribute, or not, to the literature]. Although the article possessed many strengths, including xxx, my enthusiasm for this work is limited by the following issues.”

• Identify positives

It is nice to use the first paragraph, or a very short second paragraph, to identify positives of the work. Don’t spend a lot of time on this, but if there are things that you think are noteworthy or contribute to the strength of the paper, it’s nice to include them.

The rest of the review should proceed in bulleted or short paragraph form.

The paragraphs should be comprised (roughly) of the following information:

• Clearly state each issue, starting with the most critical (most severe threat to validity).

What limits your confidence in the inferences that are being drawn from the findings? Think about the reliability and validity of the measures. Think about generalizability. Think about plausible alternatives, and provide them.

• Why is this a problem? Be specific.

Clearly explain why the issue you’ve raised is an issue. Be as concise as possible, providing relevant citations as needed. If you reference another paper in supporting your argument, provide the citation for paper so the authors can find it.

• What can be done about it?

This is quite possibly the most important part of the review. Even if the paper is rejected, a good review should give the authors an idea of how to improve their work. Authors will incorporate the reviews they get into their work even when submitting to a new journal, so do your best to make specific suggestions about how to improve the paper (e.g., inclusion of additional literature, alternative analytic approaches, etc.).

• Provide a summary paragraph only if useful.