Modeling Interaction via the Principle of Maximum Causal Entropy

Brian D. Ziebart, J. Andrew Bagnell, and Anind K. Dey
To appear at the International Conference on Machine Learning (ICML 2010).
[pdf]

Abstract: The principle of maximum entropy provides a powerful framework for statistical models of joint, conditional, and marginal distributions. However, there are many important distributions with elements of interaction and feedback where its applicability has not been established. This work presents the principle of maximum causal entropy -- an approach based on causally conditioned probabilities that can appropriately model the availability and influence of sequentially revealed side information. Using this principle, we derive Maximum Causal Entropy Influence Diagrams, a new probabilistic graphical framework for modeling decision making in settings with latent information, sequential interaction, and feedback. We describe the theoretical advantages of this model and demonstrate its applicability for statistically framing inverse optimal control and decision prediction tasks.

Related Research Areas: Inverse reinforcement learning, inverse optimal control, imitation learning, apprenticeship learning, structured prediction, conditional random fields.

Bibtex:

@inproceedings{bziebart2010modeling,
   author = {Brian D. Ziebart and J. Andrew Bagnell
            and Anind K. Dey},
   title = {Modeling Interaction via the Principle of Maximum Causal Entropy}, 
   year = {2010},
   booktitle = {ICML},
   pages = {1255--1262}
}