Multi-Agent Learning





The issue of learning and adaptation in multi-agent systems has been given increasing attention in artificial intelligence research. It is becoming clear, given the dynamic environments in which we want our agent teams to interact, that behavioral repertoires and activities cannot simply be defined in advance. Our approach to multi-agent learning, unlike the top-down model of assuming an agent's state in advance, is notable for its similarity to the types of learning exhibited by lower animal societies.

Research Goal

Our research goal is to enable multiple agents to learn a cooperative and coordinated behavior in a dynamic environment using reinforcement learning.


  • An agent does not have any prior knowledge.
  • An agent is a self-interested entity and behaves to achieve maximum reward in the range of its knowledge of the environment.


Our method is Profit Sharing Plan (PSP), which is a type of reinforcement learning algorithm. The PSP algorithm allows an autonomous agent to learn a behavior progressively without any instruction and only with delayed rewards. PSP differs from other approaches to learning (like Markov Decision Processes) in that it does not assume an agent's state in advance.


  • A. Ankolekar, Y. W. Seo, and K. Sycara, "Investigating Semantic Knowledge for Text Learning," ACM SIGIR-2003 Workshop on Semantic Web, Toronto, Canada, August 1, 2003.
  • P. Huang and K. Sycara, "Multi-agent Learning in Extensive Games with Complete Information". In Proceedings of the Second International Conference on Autonomous Agents and Multiagent Systems, Melbourne, Australia, July 14-19, 2003.

  • M. Glickman and K. Sycara, "Evolutionary Search, Stochastic Policies with Memory, and Reinforcement Learning with Hidden State," in Proceedings of the Eighteenth International Conference on Machine Learning, 2002.
  • D. Zeng and K. Sycara, "Effects of Learning in Negotiation," in Encyclopedia of Computer Science and Technology, Allen Kent and James Williams (eds), Vol 44, pp. 15-33, Marcel Dekker Inc., 2001.

  • S. Arai and K. Sycara, "Credit Assignment Method for Learning Effective Stochastic Policies in Uncertain Domains," Proceedings of Genetic and Evolutionary Computation Conference (GECCO-2001), (2001).

  • S. Arai, K. Sycara, and T. R. Payne, "Experience-based Reinforcement Learning to Acquire Effective Behavior in a Multi-agent Domain," Proceedings of the 6th Pacific Rim International Conference on Artificial Intelligence, Lecture Notes in Artificial Intelligence 1886, Springer-Verlag, pp.125-135 (2000).

  • Arai, S., and Sycara, K., Effective Learning Approach for Planning and Scheduling in Multi-Agent Domain, Proceedings of the 6th International Conference on Simulation of Adaptive Behavior (From animals to animats 6), pp.507-516 (2000).

  • S. Arai, K. Sycara and T. R.Payne "Multi-agent Reinforcement Learning for Scheduling Multiple-Goals," in Proceedings of the Fourth International Conference on Multi-Agent Systems (ICMAS'2000).

  • S. Arai, S., and K. Sycara, "Effective Learning Approach for Planning and Scheduling in Multi-Agent Domain." In Proceedings of the 6th International Conference on Simulation of Adaptive Behavior (From animals to animats 6), pp.507-516 (2000).

  • D. Zeng, and K. Sycara, "Bayesian Learning in Negotiation," in International Journal of Human Computer Systems, Vol. 48, pp.125-141, 1998.

  • D. Zeng, and K. Sycara, "Benefits of Learning in Negotiation," in Proceedings of AAAI-97 (in pdf)

  • K. Sycara, and A. Pannu, "A Learning Personal Agent for Text Filtering and Notification," in Proceedings of the International Conference of Knowledge-Based Systems (KBCS 96), Dec. 1996. (in pdf)

  • K. Sycara and A. Pannu, "Learning Text Filtering Preferences," Symposium on Machine Learning And Information Access. AAAI 96 Symposium Series, Mar. 1996, Stanford, CA. Figure 2. (in pdf)

  • K. Sycara and K. Miyashita, "Learning Control Knowledge through Case-Based Acquisition of User Optimization Preferences," in Knowledge Acquisition and Machine Learning: An Integrated Approach, Y. Kodratoff and G. Tecuci (eds), Morgan Kaufman Publishers, 1995.

  • K. Sycara, "Machine Learning for Intelligent Support of Conflict Resolution," in Decision Support Systems, Vol. 10, pp.121-136, 1993.

For more information on multi-agent learning, a summary of the PSP method, and some experimental results, see the following PowerPoint presentation:



Copyright 2006 - 2012 © Advanced Agent-Robotics Technology Lab - The Robotics Institute - Carnegie Mellon University

Internal Site (Restricted Access).