Self-rankings as a predictor of scientific impact beyond peer review.

Abstract

Peer review in academic research aims to ensure factual correctness and identify work of high scientific potential, but rapid submission growth has made this increasingly difficult. Here we investigate an additional measure for identifying high-impact research: authors' rankings of their own submissions to the same artificial intelligence (AI) conference. Grounded in game-theoretic reasoning, we hypothesize that self-rankings are informative because authors understand their work's conceptual depth and long-term promise. We tested this hypothesis in a large-scale experiment at a leading AI conference. Over more than a year, papers ranked highest by their authors received twice as many citations as their lowest-ranked counterparts, and self-rankings were especially effective at identifying highly cited papers. Self-rankings also outperformed peer-review scores in predicting future citation counts. These findings show that authors' self-rankings can complement peer review when identifying high-impact AI research.