[This is a copy of my blogpost, previously on thespectroscope.com.]
Though academic journals provide an important service as a means to select articles, sort and categorize the scientific output, it is bad (and somewhat unethical) that the whole process of scientific debate depends exclusively on an intermediate (the editor) that has certainly (and quite understandably) less thought about possible problems of an article than its authors or the peers challenging it. There must be a scientific public place where scientists can meet and debate openly about their articles in a way which goes beyond (and deeper than) the binary question of accepting/rejecting it and where new versions could be uploaded according to peers' feedback. The scientific truth should not be decided through an unverifiable process but emerge from open consensus. This scientific public place should also have a native system of blogs, well connected to articles, so that any scientist may voice and elaborate on his particular views of his/her field, which may be contradictory with others'. There is a readily available platform for that matter: sjscience.org . (post by Michaël Bon)
In 2006, a badly flawed paper on a subject that I know pretty well was published in PLoS Biology. I was part of a six-person team who wrote a rebuttal, which PLoS Biology rejected as insufficiently novel. Appeals were written and rejected. All six of us eventually found more rewarding things to do and drifted away from the project. So the original paper still stands unchallenged.
These days, I wouldn't waste my time on that process, which at best would have resulted in a response being published six months after the event when everyone had forgotten about it. I'd just go right ahead and blog about it. There's no argument to be had here: blogs are simply a much, much better medium for carrying on a dispute than peer-reviewed papers. They move faster, they're more interactive, and crucially they can't be blocked.
I hear that some of these post publication discussions erupt on blogs because the journals refuse to consider and publish comments for the published papers.
If I write a review for a paper, I own the review. I refuse to give intellectual rights and ownership to the editors/journals. I better be able to stand up and take responsibility for my reviews and opinions. Since any review I write reflects my opinions, I see no reason not to be able to share my reviews/opinions with the world on say pubmed commons. Whether I was asked by an editor or not to read the paper seems insignificant to me. The important thing is that I read the paper, I have an opinion, it is well considered, and I am willing to take full responsibility for it!
I think I have a unique perspective on this because I see how reviews work in both the academic world and in the (academic) software world, where I've been for the past 8 years. It's exceedingly rare that someone gives a heads-up to the author when they're about to write a blog post about a piece of software. Generally, the time from the inspiration for the post to the time when you start writing is hours to days. The higher-profile the reviewer, the more likely they are to give a heads-up. For example, if you are interviewed and give some critical comments to a major media outlet, you'll may get a heads-up, but often only if you ask for it - by no means is this standard or common. If you're a senior VC, you might have your post read by a few other people before posting. Fred Wilson (at http://avc.com ) often does this.
In this sense, it's a technology problem. Tech companies all have systems for monitoring what gets said about us, with the assumption that not everyone will let us know. Often, people who are commenting are speaking to a very different audience and it might not even occur to them to invite you to comment. They may want to have a discussion among their group and feel like if you jumped in at the start, it would squelch the thread. I know that I always avoided jumping in too early in discussion threads about Mendeley that I found "in the wild" for exactly that reason.
So in that context, though it seems reasonable, it's unrealistic for any author to expect prior notice of coming criticism. It's also not scalable. If everyone asked everyone before releasing anything they said about anyone, nothing would ever get written. What you're essentially asking, when you ask for prior disclosure of criticism, is just to start the discussion earlier among a smaller group. This is often done to try to keep any possible mistakes you made under wraps. I think there are good arguments to be made for doing so, if you think the reviewer is misunderstood but has a way bigger reach than you do and thus the misunderstanding will become the consensus understanding, but I would suggest that in most cases that's not the dynamic and if your reviewer really is hostile and misunderstood, public reviews give you a perfect opportunity to show the world how misunderstood they really are. Where this dynamic is worst is when a senior scientist has some criticism of a person who's very much the junior.
With respect to peer review submitted in confidence, I would agree it may not be appropriate to publish your review verbatim ahead of the paper coming out, but once the paper is out, it's fair game and you should rewrite, if necessary, the substance of your review on Pubmed Commons or your blog or PubPeer or wherever you think is best. It's just common sense courtesy that you would invite the author to respond, but a heads-up is not practical in many cases.
Essentially this is a discussion about community norms, so I would suggest the following:
1. Inspiration and time is fleeting, so have a bias towards saying something rather than not.
2. Don't be an asshole.
3. Forgive other people who are assholes to you.
4. Review unto others as you would have them review unto you. (credit @ShitAcademicsSay)
5. Expect & forgive incompleteness - perfection is unattainable - do the best you can in the time you have.
Practical example: this comment itself is poorly organized, incomplete, and I have to fight the urge to reorganize it. If I invested that time, I would then have to fight the urge to just blog my (then) essay on the topic. However, I'll have to trust that anyone who reads this will give me the benefit of the doubt, because I now have other stuff I have to go do, and I'd never get around to polishing it up and posting it. The same thought process is behind pretty much any review anyone would write. Bias towards action!
To be honest, I'm surprised there is so much debate around this. Overall, I agree with your considered thoughts.
It is pretty unpleasant if you hear on the grapevine that someone has criticised your work, but I don't think that means people shouldn't criticise it. I think, in general, it is a good idea to contact the author to let them know the critique is coming; they won't like it, but at least they won't hear through a third party. But there should be no expectation that the author has a right to veto publication of a critique. It's important to realise that, although it is good to be polite, politeness should never trump concern for accuracy. This is science, not a dinner party.
Once a paper is published, I wouldn't see any reason to hide my appraisal of it, just because the review was originally solicited by a journal editor. The example I gave of this is of a paper in PLOS One. I had reservations about the paper and expressed them in my review, but the paper was published anyhow. I don't think I would have taken further action, except that I saw that Jon Brock had published his reviewer comments on the PLOS One comment site - and he had made a number of points similar to those I made:
The authors did respond to Jon's comment, but in a way to imply they had addressed the problems. I didn't think they had, and felt I should add my voice to Brock's, so I also published my review as a comment in the same thread. I felt this was important because this was a paper by an influential author focussing on a serious developmental condition, and my impression was that there had been p-hacking and the results would not be reproducible.
My strong preference these days is to publish in the handful of journals that publish the peer review alongside the paper. PeerJ is exemplary in this regard. One reason I like this is because PeerJ is not well known and if people hear you have a paper there, they tend to assume it hasn't been reviewed properly. I like people to be able to see that the reviews are there, and, in my experience are typically as robust as anywhere else. I also use PubMed Commons increasingly to raise concerns about papers - though in general these are not papers I have reviewed. But I would not hesitate to write there if I felt an important reviewer comment had been overlooked.