Vincent Conitzer
Faculty, PI
Vincent Conitzer is a Professor of Computer Science (with affiliate/courtesy appointments in Machine Learning, Philosophy, and the Tepper School of Business) at Carnegie Mellon University. He directs the Foundations of Cooperative AI Lab (FOCAL) at CMU, whose goal is to create foundations of game theory appropriate for advanced, autonomous AI agents -– with a focus on achieving cooperation. (More detail at the FOCAL site.) Vince has also done early work on aligning (narrow) AI systems, and has various other recent interests related to the alignment of future AI systems such as the shutdown problem, aligning AI that understands the world more deeply than we do, what LLMs tell us about ourselves, and even AI consciousness.
Aran Nayebi
Faculty
Aran Nayebi is an Assistant Professor at Carnegie Mellon University’s Machine Learning Department, a core faculty member of the Neuroscience Institute, and holds a courtesy appointment in the Robotics Institute. His lab, the NeuroAgents lab, works at the intersection of neuroscience & AI to reverse-engineer
animal intelligence and build the next generation of autonomous agents, safely and responsibly.
Aydin Mohseni
Faculty
Aydin Mohseni is a scientific philosopher and assistant professor in the Department of Philosophy at Carnegie Mellon University. His research centers on two questions: how scientific communities produce knowledge, and how agents represent the world, value outcomes, and act. His work on agency focuses especially on artificial intelligence, including the foundations of AI safety, agency, and alignment. He uses Bayesian inference, decision and game theory, network theory, and agent-based modeling to study problems ranging from scientific funding and reform to causal reasoning and value learning in artificial agents. He is also a member of CMU’s Institute for Complex Social Dynamics.
Bryan Wilder
Faculty
Bryan Wilder is an Assistant Professor in the Machine Learning Department at CMU, where he leads the Lab for AI and Social Impact. The lab develops foundations for safe AI systems and collaborates with partners like governments and health systems on their deployment and evaluation. A recent focus of his work is developing a "behavioral science" for AI models: understanding how models form beliefs, make decisions, experience incentives, and so on, and how these dispositions contribute to safety-relevant behaviors.
Sven Neth
Faculty
Sven Neth is Assistant Professor of Philosophy at the University of Pittsburgh. He works on decision theory and formal epistemology with applications to AI safety, for example modeling what happens when we optimize proxy utility functions.
Nihar Shah
Faculty
Nihar B. Shah is an Associate Professor in the Machine Learning and Computer Science Departments. His research is on evaluation of science, including safety and alignment of autonomous AI scientist systems.
Derek Leben
Faculty
Derek Leben is Teaching Professor of Business Ethics at the Tepper School of Business. He teaches courses on ethics and technology, including the undergraduate course "Ethics of Emerging Technologies" and the MBA course "Ethics and AI." Dr. Leben's research focuses on principles of fairness and safety, and how organizations can make use of traditional ethical theories to develop and justify standards for responsible usage of AI. He is author of the books Ethics for Robots (Routledge, 2018) and AI Fairness (MIT Press, 2025). Course syllabi and samples of published work can be found at: derekleben.com.
Sarah Cen
Faculty
Sarah Cen is an Assistant Professor at Carnegie Mellon University in the Departments of Electrical & Computer Engineering and Engineering & Public Policy. Her research lies at the intersection of machine learning and AI accountability, drawing on tools from machine learning, statistics, and causal inference as well as concepts from law and policy. Her goal is to improve the responsible development and deployment of AI using both technical and legal perspectives. Recently, her work has studied the statistical foundations of AI auditing, the development of AI across an expanding supply chain, information and resource gaps in AI accountability, frameworks for taxation in the AI industry, and LLM behavior during election season. This work builds on conceptual thinking of AI safety by developing frameworks for thinking about AI, its role in society, its transformative impact, and incentives needed for responsible AI.
Maarten Sap
Faculty
Maarten Sap is an Assistant Professor at Carnegie Mellon University's Language Technologies Institute (LTI), with a courtesy appointment in the Human-Computer Interaction Institute (HCII). He leads the Sapling research group and is also a part-time Senior Research Scientist and Technical AI Safety Lead at the Allen Institute for AI (Ai2).
Research interests and lab goals: His research focuses on developing socially intelligent, culturally responsible, and safe AI systems. Drawing on natural language processing, social psychology, and human-computer interaction, his group studies how AI systems understand and navigate social interactions, how language technologies can perpetuate social biases and inequalities, and how AI can support positive human outcomes. A central goal is to develop better methods for evaluating and improving AI systems' social reasoning, safety, and alignment with diverse human values.
Research related to conceptual AI safety: His work examines fundamental questions about what constitutes safe and socially appropriate AI behavior with respect to user interaction, whose values AI systems should reflect, and how safety can be meaningfully evaluated across different social and cultural contexts.
This includes research on safety and privacy in tool-using agent interactions, cultural adaptability and pluralistic values, and biases in AI alignment and reward models.
His research also investigates human-centered risks of AI, including over-reliance and hidden manipulation, with the broader aim of developing AI safety frameworks that account for the complexity of human social interactions.
Andrew Ilyas
Faculty
Andrew Ilyas is an Assistant Professor at CMU. His work focuses on improving the predictability, reliability, and safety of large-scale AI systems by trying to uncover general principles that govern their behavior.
Simon DeDeo
Faculty
Simon DeDeo is a Professor in Social and Decision Sciences and leader of Proofs & Reasons.
Proofs and Reasons asks basic questions about one of our most long-running and fundamental human activities: mathematics, mathematical thinking, and mathematical proof. What is mathematics? How do we do it? How will AI change it?
Proofs in Practice: the cognitive science of how humans discover and make sense of mathematical proofs. How do we prove things?
Transcendental Structures: the formal study of mathematical proof itself (complexity, type theory, metamathematics, logic). What is the space of mathematical truth?
Cyborg Proofs: the use of AI to discover and verify proofs, with and without human aid. What tools can we build, and how will they alter mathematics?
Ida Mattsson
PhD Student
Ida Mattsson is a PhD student in the Philosophy department; Logic, Computation and Methodology program. She is interested in questions of alignment and safety from the perspective of how AI systems shape interactions and collective functions. She also helps organize the Carnegie AI Safety Initiative.
Alexander Heckett
PhD Student
Alexander Heckett is a PhD student in the Computer Science department studying under Vincent Conitzer. He studies scalable oversight over graphical models, working towards an understanding of how the structure of graphical representations of arguments influences the performance of debate protocols over them. He also studies multi-agent incentive structures in AI-for-math domains.