What an interesting experience it has been working as an academic reviewer. The papers in question have already been blind reviewed, and my job was to take a bunch of them and make sure that the second submissions had addressed the comments of the first reviewers.
They had, where those comments were reasonable. I was surprised at how many of the reviewers thought that it was OK to spit quite venomously at the authors. Some of this will be the result of anonymity, of course. You can feel safe being quite rude to someone who doesn't know who you are.
But most interesting to me was the number of pieces I read today in which a reviewer didn't actually read what the authors' intentions were, but felt free to point out the perceived shortcomings anyway. One, writing on an interesting philosphical piece, said "I'm not clear what the authors actually DID... " Well, dude, what they did was read, then think, then write. Maybe you could take a lesson from that, hmmm? Another reflective piece by two young academics (I know that they are both PhD students) on their first semester's teaching was criticised for not providing 'hard evidence' (of what? their reflections?) and not describing an LMS that they didn't use. Then there was the qualitative project, clearly explained in detail, which was criticised for a) not reporting its quantitative findings and b) not reporting on a method it didn't use.
It was illuminating. I will certainly be very well prepared for any inadequate reviews I get in the future and will take a leaf out of the book of the young academics, who replied that the reviewer was "irresponsible and unjustified" (emphasis in original). Of course all of the critiques have to be considered, but they don't all have to be addressed. Although I do understand that a mistaken reviewer might point to a weakness in an explanation.
