Ivan Stelmakh

I'm a third-year PhD student in the Machine Learning Department at CMU advised by Nihar Shah and Aarti Singh. You can find my CV here.

My research interests lie in the field of Human-AI collaboration in peer-assessment tasks, with a current focus on academic peer review. The rapid growth of the number of submissions to leading publication venues has identified a need for automation of some parts of the peer-review pipeline. However, there is evidence that application of off-the-shelf technologies may amplify unfairness of the process and lead to other unintended consequences. In my research, I aim at identifying and overcoming such unintended consequences by designing tools and techniques to support fair, equitable and efficient Human-AI collaboration in peer review.

Before coming to CMU, I received a B.S. in Applied Mathematics and Physics from Moscow Institute of Physics and Technology, where I was adviced by Vladimir Vyugin.

Email: stiv [at] cs.cmu.edu

News

March 2020
I joined the editorial team of CMU ML Blog. If you are a CMU student, postdoc or faculty, consider featuring your work in our blog!

December 2019
I gave a talk at UWaterloo and a spotlight presentation at NeurIPS.

October 2019
My team took the second place at Citadel East Cost Regional Datathon!

September 2019
Our work on testing for biases in single-blind peer review is selected for a spotlight presentation at NeurIPS 2019!

August 2019
A follow-up paper (Kobren et al., 2019) to our PeerReview4All work came out at KDD. Authors introduce new assignment algorithms and compare them with our PeerReview4All algo on the real conference data. The results seem very promising as PR4A achieves the best balance between the fairness (Min PS) and the average quality of the assignment (Mean PS).

August 2019
I'm excited to join ICML2020 program committee as a workflow chair!

June 2019
I was fortunate to attend the ACM EC Workshop on Mechanism Design for Social Good, where I presented our work on fair peer review and got to know many remarkable works done by other participants!


Publications

On Testing for Biases in Peer Review
Ivan Stelmakh, Nihar Shah and Aarti Singh

NeurIPS 2019
PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review
Ivan Stelmakh, Nihar Shah and Aarti Singh
Code for the PR4A assignment algorithm
Dataset

ALT 2019
Adaptive Algorithm of Tracking the Best Experts Trajectory
Vladimir Vyugin, Ivan Stelmakh and Vladimir Trunov

JCTE