Comparison with subjective learning of convex sets of distributions

 

This paper concentrates on the connection between data and convex sets of distributions. Results do not use the possible existence of prior distribution for events. An alternative approach is to use prior distributions and Bayes rule to obtain posterior measures for events. The objective of this paper is not to replace Bayes rule, but rather to enhance one's intuition about probabilities constructed solely from data. There has been work on subjective approaches to the process of learning convex sets of distributions; we mention two approaches that are relevant to Bayesian networks.

Ramoni and Sebastiani approach to missing data

The estimation of parameters for a Bayesian network usually has to deal with missing data, i.e., observations for some variables are not collected. The standard Bayesian assumption is that missing data happens at random; if this assumption is violated, inferences may be biased. Ramoni and Sebastiani propose to lift the ``missing at random assumption'' [19] in a Bayesian network learning scenario. They consider all possible ways in which missing data could have happened, and create a convex set of joint distributions that represent the gamut of possibilities for the data actually collected. The idea is to avoid using unjustified assumptions and replacing those by sets of distributions, so that the effects of missing data can be evaluated.

Walley's imprecise Dirichlet prior

The imprecise Dirichlet prior has been proposed by Walley [26] as a model for inferences associated with multinomial sampling. Here we indicate how this model can be used to learn Bayesian networks associated with convex sets of distributions.

An imprecise Dirichlet distribution for a vector valued variable theta is:

p(theta) = Dir(theta| s, t) prod_i=2^|theta| theta_i²s t_i - 1,

where s is a real number larger than zero and t is a vector where sumt_i = 1 and 0 < t_i < 1 for all t_i.

This class of distributions can be used as a prior credal set; the prior assumptions are much less restrictive than standard Bayesian assumptions. Note that for any event A, the prior imprecise Dirichlet model induces the bounds p(A) = 0 and p(A) = 1.

First consider standard Bayesian network learning when complete data is available. A Bayesian network codifies a joint distribution through the expression:

p(x) = prod_i p(x_i | pa(x_i)),

where pa(x_i) are the parents of variable x_i. For each variable, the vector of parameters theta_i contains elements theta_ijk = p(x_i = k| pa(x_i) = j), where theta_ij1 = 1 - sum_k=2²|x_i| theta_ijk. The vector theta_ij = { theta_ijk }_k=1²|x_i| contains the relevant parameters for the distribution p(x_i | pa(x_i) = j). The vector Theta= { theta₁, &ldots;, theta_n } contains all parameters to be estimated. The usual assumption for the prior p(Theta) is parameter independence:

p(Theta) = prod_i=1ⁿ prod_j=1³pa(x_i) p(theta_ij).

Finally, the prior distributions for each vector theta_ij are assumed to come from an imprecise Dirichlet family. The posterior is then an imprecise Dirichlet distribution with parameters that depend on the prior parameters and the data.

Suppose that every vector theta_ij is associated with an imprecise Dirichlet prior:

p(theta_ij) = Dir(theta_ij|s_ij, t_ij) prod_k=2²|x_i| theta_ijk³s_ij t_ijk - 1,

where s_ij is a real number larger than zero and t_ij is a vector such that sumt_ijk = 1 and 0 < t_ijk < 1 for all t_ijk. We assume that the convex set of prior joint distributions is obtained by taking the convex hull of all prior marginals defined by imprecise Dirichlet distributions.

Suppose data n_ij observations are made with pa(x_i) = j and n_ijk observations are made with x_i = k, pa(x_i) = j.

The posterior distribution for theta_ij is given by imprecise Dirichlet distributions, due to the parameter independence assumption and the convexification convention. We have theta_ij with marginals:

p(theta_ij = Dir(theta_ij | s_ij' , t_ij'),

where s_ij' = n_ij + s_ij and t_ijk' = { n_ijk + s_ij t_ijk }/{ n_ij + s_ij } .