I am an associate professor at CMU in the Machine Learning and the Computer Science departments. I work in the areas of machine learning, statistics, information theory and game theory. My current work addresses various biases and other challenges in human evaluations via principled and practical approaches. A focus application is scientific peer review, where our work has already made significant impact.

Survey on Systemic Challenges and Solutions on Bias and Unfairness in Peer Review, and associated tutorial slides

Google Scholar page

nihars [at] cs.cmu.edu
Office: GHC 8211



RESEARCH

My research focusses on two inter-related themes:

  1. Distributed human evaluations: Distributed human evaluations are integral to various applications where a set of items is assessed by a group of individuals, each person evaluating only a subset of the items and each item being evaluated by only a handful of individuals. This setup is commonly seen in domains such as scientific peer review, hiring and promotion processes, academic admissions, crowdsourcing initiatives, healthcare assessments, judicial verdicts, and online rating and recommendation systems, among others. The decentralization of evaluations, however, often leads to a host of challenges including instances of fraud, subjectivity, miscalibration, biases, breaches of privacy, and operational inefficiencies.
  2. Research on research Our research also focuses on problems in scientific research itself and its evaluations. The challenges within this domain hold substantial consequences. They impact the allocation of billions of dollars in annual grant awards, influence the trajectories of researchers' careers due to the "rich-gets-richer" phenomenon in academia, and can detrimentally affect the public's perception of science.
Our research addresses these important challenges at scale, in a principled and pragmatic manner. Our work is comprehensive, establishing fundamental limits, designing algorithms, deriving theoretical guaranees for them, conducting evaluations, and working with users for actual deployment and impact. The algorithms we have developed are now extensively used for peer review of tens of thousands of papers across computer science and various other fields. Additionally, our experiments have helped to shape the policies of numerous peer-review venues in an evidence-based manner. Finally, the impact of our work extends to domains beyond review of scientific papers.



PUBLICATIONS



GROUP
PHD STUDENTS


Charvi Rastogi
Machine Learning Department
(advised jointly with Ken Holstein)

Steven Jecmen
Computer Science Department
(advised jointly with Fei Fang)

Alexander Goldberg
Computer Science Department
(advised jointly with Giulia Fanti)

Keerthana Gurushankar
Computer Science Department

MASTERS STUDENTS


Ryan Liu
Computer Science

UNDERGRADUATE STUDENTS


Janet Hsieh
Computer Science

Priyanshi Garg
Computer Science

ALUMNI


Ivan Stelmakh
PhD, Machine Learning Department
(advised jointly with Aarti Singh)

Jingyan Wang
PhD, Robotics Institute

Carmel Baharav
BS in Computer Science

Komal Dhull
BS in Computer Science

Wenxin Ding
MS in Computer Science
BS in Mathematics and Computer Science
(advised jointly with Weina Wang)

Qiqi Xu
MS in Machine Learning
(advised jointly with Hoda Heidari)


FUNDING
We gratefully acknowledge support from the National Science Foundation, CMU Block center, CMU CyLab, ONR, a Google Research Scholar award, a JP Morgan Faculty Research Award, and an NSF-Amazon Fair AI research grant.


TEACHING

Fall 2023 10-715 Advanced Introduction to Machine Learning
Fall 2022 10-715 Advanced Introduction to Machine Learning
Spring 2022 15-780 Graduate Artificial Intelligence
Fall 2021 10-715 Advanced Introduction to Machine Learning
Spring 2021 15-780 Graduate Artificial Intelligence
Fall 2020 10-715 Advanced Introduction to Machine Learning
Spring 2020 15-780 Graduate Artificial Intelligence
Fall 2019 10-715 Advanced Introduction to Machine Learning
Spring 2019 15-780 Graduate Artificial Intelligence
Fall 2017 10-709 Fundamentals of Learning from the Crowd


In my spare time, I am also creating introductory machine learning short lectures in Hindi, accessible to anyone without requiring any math or programming knowledge: Link to Youtube videos


CURRICULUM VITAE
EDUCATION
HONORS

SERVICE (outside CMU)

INDUSTRY EXPERIENCE