CMU Artificial Intelligence Seminar Series sponsored by Fortive
Tuesday, May 11, 2021
Time: 12:00 - 01:00 PM ET
Vladlen Koltun -- Evaluating Researchers
Relevant Paper(s):
Abstract: The impact of individual scientists is commonly quantified using citation-based measures. The most common such measure is
the h-index. A scientist’s h-index affects hiring, promotion, and funding decisions, and thus shapes the progress of science.
Here we report a large-scale study of scientometric measures, analyzing millions of articles and hundreds of millions of
citations across four scientific fields and two data platforms. We find that the correlation of the h-index with awards that indicate
recognition by the scientific community has substantially declined. These trends are associated with changing authorship
patterns. We show that these declines can be mitigated by fractional allocation of citations among authors, which has been
discussed in the literature but not implemented at scale. We find that a fractional analogue of the h-index outperforms other
measures as a correlate and predictor of scientific awards. Our results suggest that the use of the h-index in ranking scientists
should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives.
Bio: Vladlen Koltun is the Chief Scientist for Intelligent Systems at Intel. He directs the Intelligent Systems Lab, which conducts high-impact basic research in computer vision, machine learning, robotics, and related areas. He has mentored more than 50 PhD students, postdocs, research scientists, and PhD student interns, many of whom are now successful research leaders.