Useful blog post at orgtheory.net. It consists of a series of pointers from editors at top sociology journals as to what makes for useful comments/review notes when you are asked to review an academic article.
As we in academia have to do this quite a bit, even if just informally helping out a colleague, these are some useful tips to keep in mind. A few that stuck out for me:
“Be developmental – Good reviews suggest solutions to the problems raised. Just saying, “this is bad, this is wrong, I disagree with this, why should I care…” doesn’t really help the authors. Telling them what would solve the problem for you does. If you can’t come up with a tractable solution, you may want to reconsider whether or not you are making a valid criticism, or being too negative and nit-picky. All research has weaknesses. The question is, are these weaknesses fatal, addressable, or just inherent to the theory or methodological approach employed and thus not something the author can address but should be mindful of when drawing inferences and making claims.”
“Start from a happy place – If you approach a review assuming you are going to reject the manuscript odds are you will, because you will focus primarily on information that confirms this expectation. As a reviewer, when I start reading a paper I assume that I’m going to give it an R&R unless the authors convince me otherwise. This approach makes me more open to the positive aspects of the paper and the potential nuggets that can be developed, and which might be overlooked if you are only looking for reasons to reject the manuscript. It may not take long for the authors to convince me to reject their paper, but I’ve also seen some real diamonds in the rough polished into gems because I was open to seeing what the paper might be.”