AAAI 2021, AAMAS 2021, EC 2021, IJCAI 2022 tutorial: Designing Agents' Preferences, Beliefs, and Identities


Brief description:
We often assume that each agent has a well-defined identity, well-defined preferences over outcomes, and well-defined beliefs about the world. However, when designing agents, we in fact need to specify where the boundaries between one agent and another in the system lie, what objective functions these agents aim to maximize, and to some extent even what belief formation processes they use. What is the right way to do so? As more and more AI systems are deployed in the world, this question becomes increasingly important. In this tutorial, I will show how it can be approached from the perspectives of decision theory, game theory, social choice theory, and the algorithmic and computational aspects of these fields. (No previous background required.)

A AAAI'19 blue sky writeup on a subset of these ideas can be found here: http://www.cs.cmu.edu/~conitzer/designingAAAI19.pdf See also our new Foundations of Cooperative AI Lab (FOCAL) at CMU.

Outline:

• Learning an objective from multiple people
  • Focus on moral reasoning
  • Use social choice theory
• Decision and game-theoretic approaches to agent design
  • Causal and evidential decision theory (and others)
  • Imperfect recall and Sleeping Beauty
  • Program equilibrium
• Conclusion

Slides: pptx, pdf.
The EC'21 version of the tutorial on YouTube: part 1, part 2, part 3, part 4.