• Graduate Education
  • Research
  • Browse our Courses
  • Events
    • Events Calendar
    • Events Archive
    • Summer School
      • Applied Public Policy Evaluation
      • Deep Learning
      • Development Economics
      • Economics of Blockchain and Digital Currencies
      • Economics of Climate Change
      • The Economics of Crime
      • Foundations of Machine Learning with Applications in Python
      • From Preference to Choice: The Economic Theory of Decision-Making
      • Inequalities in Health and Healthcare
      • Marketing Research with Purpose
      • Markets with Frictions
      • Modern Toolbox for Spatial and Functional Data
      • Sustainable Finance
      • Tuition Fees and Payment
      • Business Data Science Summer School Program
    • Tinbergen Institute Lectures
    • 2026 Tinbergen Institute Opening Conference
    • Annual Tinbergen Institute Conference
  • News
  • Alumni

Albert Menkveld, Charles Bos, Chen Zhou, Egle Karmaziene, Jan Wrampelmeyer, Jingni Yang, Laura Capera, Michel van der Wel, Norman Seeger, Patrick Verwijmeren, Remco Zwinkels, Rex Wang Renjie, Sebastian Vogel, Shihao Yu, Utz Weitzel and Vladimir Vladimirov et al. (2024). Nonstandard Errors The Journal of Finance, 79(3):2339--2390.

  • Journal
    The Journal of Finance
View publication

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.