ladamic's blog research on information networks and non-researchy random musings

For the past year, I have been keeping a tally of the reviewing work I’ve been doing. I’ve reviewed 81 papers & grant proposals in the past 12 months, or roughly 6.75 reviews/month. Over the same period, I’ve submitted grant proposals & papers 12 times (approx.: some submissions correspond to the same paper, ahem). So I guess I review ~7 things for others, for every item I submit for review.
Is that fair? 7:1? Could it be more like 3:1? I already seem to say no to 1/3-1/2 of the requests I get. Why do I get so many requests? Maybe it’s that as a generic “networks” person I’m of use as a reviewer on interdisciplinary panels, in CS venues, Physics venues, even some stats and social science venues. Some swear by farming our reviews to students. I’ve done that for approx 10 reviews this year (on top of the 81 I did myself). But it seems like a drop in the bucket compared to the total load.
I used to look forward to some reviewing. Getting a paper authored by a respected colleague in the field. Now I often turn those down. About a year ago, I had been asked by a prestigious journal to review a paper by someone whose work I admired and whose sense of humor I really enjoyed. But I felt that this particular paper did not pack the punch that I expected of papers in that journal, and I did not recommend it for publication (on both rounds). My name and comments to the editor (“this would be fine for a lesser journal…” (ouch!)) were accidentally released to the authors. Although I know the probability of such an event occurring again are small, even the thought of rejecting a paper by someone I chat with at conferences, or who is likely to tell me about the work even before I review it, is somehow uncomfortable.
I consider myself a middle-of-the-road reviewer. Especially with the conference reviewing systems these days where you are encouraged to look at others’ reviews of the same paper for purposes of discussion, I find myself on the accept side of the middle. Yet there are plenty of papers that I don’t recommend for publication. And it makes for sad work.
At this point I’m doing so much reviewing, that I find myself asking the authors of one paper why they didn’t cite another paper I had reviewed a short while previously. At 7 papers/proposals to review a month, I honestly don’t think I read a comparable number of papers I “want” to read or “should” read for my research. Not to mention papers given to me by colleagues, that I put on the back of my reviewing queue and often don’t get to.