A thought while procrastinating on a manuscript review…  A few years ago I published a paper which presented a solution to a problem that no one had solved before – at least not solved and then published their results.  The paper isn’t going to win me a Nobel but I believe it contains some useful takeaways.  (In addition to addressing a particular problem, it combines a couple useful mathematical methods in a way I hadn’t seen done before and which may have broader applicability.)  My motivation for submitting to that journal was that a) the journal is reasonably widely read by people in my field and b) all articles are peer-reviewed.  Both things matter to me.  The latter in particular.  It’s appropriate to have a few people look at your work with fresh eyes and verify that you’re not full of it.   (So I probably shouldn’t mention that only one reviewer provided feedback on the manuscript and, based on the speed with which he responded, I’m not convinced that he read it but I was seeking out constructive criticism even if I didn’t end up getting much.)

The paper itself addresses a niche subject area so not many citations since publication and only a few dozen downloads.  Those things noted, I am satisfied that it’s thorough and correct.   Hold that thought:  Not wildly popular but thorough and correct.  As noted above, the paper has only been cited a few times and several of the citing publications are unrefereed conference proceedings which state in euphemistic terms that my results were crap.   Not surprisingly I was unhappy to discover those publications.  The four principal reasons those publications annoy me:

1. The authors do not present an analysis to support their assertions.
2. The key figures they use to justify their conclusions strongly suggest that they analyzed their data incorrectly and smack down my work are based on misapplication of the model I presented.
3. The publications are unrefereed conference proceedings so there was/is no forum for me to rebut their claims.  (If the publications had been in a peer-reviewed journal I could have written a letter to the editor with my complaints and challenged them to defend their conclusions.)
4. I’m >90% sure that one of the authors was the reviewer who gave me half-assed feedback on my manuscript.

I was annoyed but, bearing in mind that the publications are unrefereed conference proceedings on a niche subject area, I decided that they weren’t worth losing sleep over.   Had the papers been in a peer-reviewed journal I would have responded strongly but when the stakes are low let it go.  Anyhow, fast forward a couple years.  I need to hire a couple people for my group.   The job openings get posted on-line.  The applications come in.  Guess who applies?  That’s right, the lead author of the offending publications.  No interview for you, mother@#$%&*.

ADDENDUM:  I think that the above anecdote actually ties in well with Andrew Gelman’s “Replication backlash.”