Introduction

This paper investigates the possibility of learning convex sets of probability distributions from data. Several theories of inference and decision employ sets of probability distributions as the fundamental representation of beliefs: in robust Statistics [1, 10], in relation to inner/outer measures for representation of subjective beliefs [7, 20, 24], as more flexible and general measures of uncertainty [2, 4, 5, 6, 9, 15, 21, 23, 25]. Usually such sets of distributions represent subjective opinions and preferences, and the indeterminacy of beliefs is epistemic.

Frequentist models depart from subjective interpretations and relate probability to observable phenomena, whereby an underlying probability reveals itself by way of asymptotic relative frequencies. This paper examines an analogous connection between convex sets of probability and observed outcome sequences. From an infinitely long sequence of outcomes, we attempt to recover the underlying convex set of distributions from which the data was generated. Our asymptotic results parallel and generalize the laws of large numbers used in probability theory. Existing literature does not provide an organized collection of asymptotic results for convex sets of distributions. The first results of this kind were proposed by Walley and Fine [27], and this paper can be understood as an adaptation of their results to more practical scenarios. The goals of our paper are:

1. To provide background on the theory of convex sets of distributions and motivation (Sections 2 and 3).
2. To describe a framework in which data can be viewed as being ``generated'' from an underlying convex set of distributions (Section 4).
3. To clearly define the notion of an estimator for convex sets of distributions (Section 5)
4. To describe Walley and Fine's estimator in an accessible fashion, and to improve upon it (Section 6).
5. To present new classes of estimators with asymptotic convergence results (Section 7).
6. To compare this approach to approaches that learn sets of distributions with prior subjective constraints (Section 8).

The paper presents novel asymptotic results (Section 7), which can be viewed as laws of large numbers for convex sets of distributions. The results show that by examining a finite number of subsequences of the observed trials, it is possible to learn a set of distributions that is guaranteed to dominate the set that generated the data. The theorems show how any estimator, including Walley and Fine's estimator, can be improved upon; our estimators lead to more realistic characteristics than Walley and Fine's estimator.