Machine learning is primarily concerned with the design and analysis of algorithms that learn about an entity. Increasingly more, machine learning is being used to design policies that affect the entity it once learned about. This can cause the entity to react and present a different behavior. Additionally, in many environments, multiple learners learn concurrently about one or more related entities. This can bring about a range of interactions between individual learners.
How do the learners and entities interact? How do these interactions change the task at hand? What are some desirable interactions in a learning environment? And what are the mechanisms for bringing about such desirable interactions?
The first Workshop on Learning in Presence of Strategic Behavior will be held in conjunction with the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, California, on December 8, 2017.
The main goal of this workshop is to address current challenges and opportunities that arise from the presence of strategic behavior in machine learning. This workshop aims at bringing together members of different communities, including machine learning, economics, theoretical computer science, and social computing, to share recent results, discuss important directions for future research, and foster collaborations. Papers from a rich set of theoretical and applied perspectives are invited.
Learning in Strategic Data Environments.
We live in a world where activities and interactions are recorded as data: food consumption, workout activities, buying and selling products, sharing information and experiences, borrowing and lending money, and exchanging excess resources. Scientists use the rich data of these activities to understand human social behavior, generate accurate predictions, and make policy recommendations. Machine learning traditionally take such data as given, often treating them as independent samples from some unknown statistical distribution. However, such data are possessed or generated by potentially strategic people in the context of specific interaction rules. Hence, what data become available depends on the interaction rules. For example, people with sensitive medical conditions may not reveal their medical data in a survey but could be willing to share them when compensated; crowd workers may not put in a good-faith effort in completing a task if they know that the requester cannot verify the quality of their contributions. In this talk, I argue that a holistic view that jointly considers data acquisition and learning is important. I will discuss two projects. The first project considers acquiring data from strategic data holders who have private cost for revealing their data and then learning from the acquired data. We provide a risk bound on learning, analogous to classic risk bounds, for situations when agents’ private costs can correlate with their data in arbitrary ways. The second project leverages techniques in learning to design a mechanism for obtaining high-quality data from strategic data holders. The mechanism has a strong incentive property: it is a dominant strategy for each agent to truthfully reveal their data even if we have no ground truth to directly evaluate their contributions.
This talk is based on joint works with Jacob Abernethy, Chien-Ju Ho, Yang Liu, and Bo Waggoner.
Learning with Adversaries and Collaborators.
We argue that the standard machine learning paradigm is both too weak and too strong. First, we show that current systems for image classification and reading comprehension are vulnerable to adversarial attacks, suggesting that existing learning setups are inadequate to produce systems with robust behavior. Second, we show that in an interactive learning setting where incentives are aligned, a system can learn a simple natural language from a user from scratch, suggesting that much more can be learned under a cooperative setting.
Regret minimization against strategic buyers.
This talk presents an overview of several recent algorithms for regret minimization against strategic buyers in the context of posted-price auctions, which are crucial for revenue optimization in online advertisement.
Joint work with Andres Munoz Medina.
Towards cooperative AI
Social dilemmas are situations where individuals face a temptation to increase their payoffs at a cost to total welfare. Importantly, social dilemmas are ubiquitous in real world interactions. We show how to modify modern reinforcement learning methods to construct agents that act in ways that are simple to understand, begin by cooperating, try to avoid being exploited, and forgiving (try to return to mutual cooperation). Such agents can maintain cooperation in Markov social dilemmas with both perfect and imperfect information. Our construction does not require training methods beyond a modification of self-play, thus if an environment is such that good strategies can be constructed in the zero-sum case (eg. Atari) then we can construct agents that solve social dilemmas in this environment.
Online learning with partial information for players in games.
Learning has been adopted as a general behavioral model for players in repeated games. Learning offers a way that players can adopt to (possibly changing) environment. Learning guarantees high social welfare in many games (including traffic routing as well as online auctions), even when the game or the population of players is dynamically changing. The rate at which the game can change depends on the speed of convergence of the learning algorithm. If players observe all other participants, which such full information feedback classical learning algorithms offer very fast convergence. However, such full information feedback is often not available, and the convergence of classical algorithms with partial feedback is much good. In this talk we develop a black-box approach for learning where the learner observes as feedback only losses of a subset of the actions. The simplicity and black box nature of the approach allows us to use of this faster learning rate as a behavioral assumption in games. Talk based on joint work with Thodoris Lykouris and Karthik Sridharan.
Location: The workshop takes place at room 101 A on Fri Dec 08.
The NIPS17 Workshop on Learning in Presence of Strategic Behavior will be held in conjunction with the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, California, on December 8, 2017.
The main goal of this workshop is to address current challenges and opportunities that arise from the presence of strategic behavior in machine learning. This workshop aims at bringing together members of different communities, including machine learning, economics, theoretical computer science, and social computing, to share recent results, discuss important directions for future research, and foster collaborations.
Papers from a rich set of theoretical and applied perspectives are invited. Some areas of interest at the interface of learning and strategic behavior include, but are not limited to:
We solicit submission of published and unpublished works. For the former, we request that the authors clearly state the venue of previous publication. Authors are also encouraged to provide a link to an online version of the paper (such as on arXiv). If accepted, such papers will be linked via an index to give an informal record of the workshop. This workshop will have no published proceedings. Accepted submissions will be presented as posters or talks.
Submissions are limited to three pages using the NIPS 2017 format. One additional page containing only cited references is allowed. The review process is not blind. Please use the camera- ready instructions to produce a PDF in the NIPS 2017 format that displays the author’s names.
All submissions should be made through EasyChair on or before October 23, 2017, 11:59pm AoE. Notification of acceptance will be on November 4, 2017.
Submissions will be evaluated based on their relevance to the theme of the workshop and the novelty of the work.
Please refer to the NIPS 2017 website for registration details. We encourage the audience to register for the workshops as soon as possible, as only a limited number of registrations may be available.
Please direct your questions and comments regarding the workshop to email@example.com.