How Not To Review A Paper

On occasion of several paper reviews I've received recently, and a few I've written, I'd like to give some useful tips for how to review a paper. That is, how to review a paper if you want to do a really, really bad job of it. Note that I work in the AI and Games field, so somewhat different advice might apply to screwing up a paper review in another field.

First of all, be vague. Say what you think about the paper in very abstract terms, and at all costs avoid pointing out specific flaws with the paper so that they could be easily fixed.

This applies most of all to any comments about the literature review. It's fine to point out that the literature review is missing important related work, but by no means include any references to said work. Ideally, say that the paper cannot be accepted because of glaring omissions in the references, and fail to provide a single paper they should have referenced.

If by any chance you think the paper you are reviewing should have referenced one of your own papers, then you should definitely not say so. Your papers are obviously of such brilliance that everybody already know about them by virtue of them being published somewhere. Instead, treat this omission of citation as a personal insult, and add a passive-agressive slant to your review.

If you find it hard to be abstract enough in your review, then you may consider doing the opposite: only talk about details. Talk at length about verb forms and the possible inclusion of semicolons, and if you have substantial comments about the methodology or results bury them as deep as you can in a wall of text. For maximum effect, use a stream-of-consciousness style where you jot things down as you read the paper, often digressing into reflections on various topics that reading the paper reminded you of. It's great if some of your later comments contradict your earlier comments. At the end of it all, issue an arbitrary accept/reject recommendation without explaining which of the numerous comments made you come to this conclusion.

It's imperative that you don't write any summary of your review, or the effect is lost.

If the language and tone of the paper is not exactly how you would have written it, urge the author(s) to enlist the help of a native English speaker to correct their English. This comment works best if you can tell that the authors are native English speakers, and if you carefully add some grammar and spelling mistakes to your review.

Speaking of how you would have written the paper, it's a good idea to evaluate the paper from the perspective of what you would have done if you wrote the paper. Say for example that the authors present an algorithm and are mostly interested in the algorithm's correctness, whereas you would personally be more interested in its runtime. Then it's perfectly fine to reject the paper because they studied the correctness and not the runtime of the algorithm, and they clearly should have studied the runtime instead. After all, you are the one reviewing the paper, so you should decide what it should be about.

In the same vein, don't just accept any definitions of terms or scoping of the investigation that the paper might contain. Read the paper using whatever meaning of the words in it that you find convenient. And if the authors state that they are not concerned with topic X, that is no reason for you to not go on at length about how important topic X is and why they should have included it.  It's your freedom to read the paper any way you want and assign any kind of meaning to it you like.

This brings to our final and perhaps most important piece of advice. It's likely that there is some part of the paper you don't understand, because like everybody else you are occasionally frequently out of your depth and these authors write like morons anyway. If this happens - don't admit it! Don't lose face by explaining that you don't understand the paper! Your reputation as an anonymous reviewer is at stake. Instead, simply pick at random some interpretation of the part of the paper you don't understand, preferably some interpretation that makes very little sense. Then write your review as if that interpretation was true. Hilarity ensues, at least on your side.

If you heed all this advice, you will surely be able to produce the kind of reviews that one frequently receives after submitting to some famous conferences and well-respected journals.