After co-chairing a few conference tracks, I now have a little bit of experience in herding reviewers (getting them to turn in or improve their referee reports). Conference reviewing is no day at the beach (though it can be, depending on what kind of beach reading you prefer): reviewing 5-10 papers, usually the day before the deadline.
I’ve encountered a few types of reviewers:
- The ideal reviewer: turns in their reviews on time, provides detailed, critical, constructive comments.
- The superficial reviewer: submits short, usually positive reviews, with a few requests to cite their own work
- The slightly tardy, near-ideal reviewer: emails you that his/her reviews will be one day late. Detailed, critical, constructive reviews are submitted one day late.
- The con: does not submit all reviews. After repeated pleas, tells you his/her reviews are essentially done and he/she will upload them soon (today, tomorrow). Two weeks later no reviews are submitted, but you still want to believe their lies.
Here are a few amusing emails from cons (I wish I had kept them all, but I may have deleted some in irritation ):
6/30/2008 email: “… dear Lada, thanks for the invitation to join the PC …, which I accept with enthusiasm. I will be able to allocate enough time for the review process and the subsequent discussion. ”
[deadline for reviews is 12/17/2008]
12/21/2008 email: “Dear Lada, thanks for the patience… I plan to enter my reviews tomorrow morning so that you should find most of them in Easychair by the time it dawns in midwest.”
1/5/2009 email (after repeated nagging and no submitted reviews): “Lada, … I am posting them to easychair as I write this. ”
[Review deadline was 12/5/2010]
12/10/2010 email: “Hi Lada, I was busy with a proposal till yesterday, and this just slipped. I got the reviews done this morning, and will upload them in the next couple of hours.”
12/16/2010 email: “Lada, I have the reviews. One of my students is putting them into the right format, and should be able to send you soon.”
[It is now 12/20 and the reviews are still not submitted]
It is understandable that sometimes circumstances arise that prevent a PC member from reviewing. But if this is the case, please don’t tell me you have the reviews. Tell me what’s up. Tell me you can’t do it. I’ll re-assign the papers!