AAMAS 2025 tutorial: Foundations of Cooperative AI



Brief description:
Presenters: Vincent Conitzer and Caspar Oesterheld.

Target audience/expected background: The tutorial should be accessible to the AAMAS audience at large; in particular, we will not require background in game theory. The audience should be familiar with probability and very basic decision theory (maximizing expected utility). While attendees who do have background in game theory will probably catch on a bit quicker especially early in the tutorial, there are more than enough nonstandard ideas here from a game-theoretic perspective that they should find plenty that is new.

Learning outcomes/takeaway skills: The main takeaway is an introduction to the emerging area of cooperative AI -- standard game-theoretic solution concepts and approaches towards achieving cooperation in game theory, other approaches informed by other disciplines (ethics, social networks, etc.), the philosophical foundations of cooperative AI with technical consequences, its practical aspects, and community resources.

Brief description: AI systems can interact in unexpected ways, sometimes with disastrous consequences. As AI gets to control more of our world, these interactions will become more common and have higher stakes. As AI becomes more advanced, these interactions will become more sophisticated, and multiagent systems and game theory will provide the tools for analyzing these interactions. In particular, bad interactions may be driven by game-theoretic dynamics, such as the tragedy of the commons. However, AI agents are in some ways unlike the agents traditionally studied in game theory. This introduces new challenges, as human ways to maintain cooperation may not apply; but it also introduces new opportunities, as we can design agents in ways that facilitate cooperation and trust, for example by making their reasoning transparent to other agents. These developments have led to the nascent area of cooperative AI. In this tutorial, we will give an introduction to the foundations of this area and lay out future directions for research (no previous background required).

A AAAI'23 blue sky writeup on a subset of these ideas can be found here: https://www.cs.cmu.edu/~conitzer/FOCALAAAI23.pdf
See also our Foundations of Cooperative AI Lab (FOCAL) at CMU.

Slides: Part I (pptx, pdf). Part II (pdf).