Benlog

security, privacy, transparency.
« Enough with Secrecy in Research
The Beautiful Magic of Cryptography »

Luis von Ahn: make academic reviews public

Filed under: uncategorized — March 1, 2009 @ 5:36 pm

Yes! Luis von Ahn says that academic paper reviews should be public (they can remain anonymous.) I agree.

I’d go further than Luis. For most computer science conferences, there is no feedback loop. Want to trash a paper? Write a really bad review and argue strongly, and if someone else on the program committee doesn’t want to fight you, then the paper goes into the trash bin and that’s the end of that. If authors had the right to respond and to point out shoddy review work, it might make reviewers think twice about doing a poor job. As Luis points out, if we had a ranking of the reviewers and not just of the submitters, the quality of reviewing might improve over time.

  • http://www.cs.rice.edu/~dwallach/ Dan Wallach

    On the other hand, people will be less willing to be a reviewer.

    What I’d like to see is a system where you can submit your paper along with reviews from an earlier rejection where, just like a journal paper, you can talk about how you addressed the reviewer’s comments.

    As long as we’ve got papers getting rejected from one conference and resubmitted to others, we might as well create a mechanism (even if it’s optional) for the authors to carry forward the knowledge from one PC to the next.

  • http://ben.adida.net ben

    Dan,

    True, we might have to make reviewing one of the criteria for getting tenure!

    Carrying forward reviews is a nice idea. Then you and I can come up with a digital signature system so that you can prove that these were indeed the comments from the previous conference :)