Top Researchers as Academic Evaluators

Abstract

Peer review depends on expert participation, yet those best equipped to evaluate others often face the highest opportunity costs. This paper examines how evaluator quality shapes academic assessments, how effort is adjusted under time constraints, and what this implies for the sustainability of evaluation systems. Using rich administrative data from Italy’s Abilitazione Scientifica Nazionale (ASN) – a centralized promotion system where evaluators are randomly assigned to national field-level committees – we show that more productive researchers apply stricter standards, place greater weight on publication quality over quantity, and their assessments more accurately predict candidates’ future research performance and career advancement. Fields assigned stronger committees experience a temporary decline in low-quality (predatory) publications, suggesting that the quality of evaluators influences broader academic incentives. Despite apparent effort optimization, serving on ASN committees – typically a two- year appointment involving the evaluation of over 400 applications – imposes a sizable research cost on top researchers, equivalent to 30% of a year’s output. Over time, they become less likely to volunteer, revealing a structural tension: systems that improve evaluation quality by relying more on expert reviewers may ultimately reduce their willingness to participate, making sustained excellence an elusive target. Joint paper with Natalia Zinovyeva.