Sampling-based techniques

 

Once we recast the robust inference problem as the estimation of parameters Theta, we can also assume the parameters Theta to be assigned uniform priors. In this case, any value of Theta that maximizes p(x_q = a, Theta|e) also maximizes p(x_q = a|e, Theta). We are interested in algorithms that produce such maximizing values of Theta, since p(x_q = a|e) = max_Theta p(x_q = a|e, Theta). With this maneuver, we can use Bayesian learning methods to produce robust inferences.

Sampling algorithms for calculation of posterior maxima have been studied in connection with Bayesian inference in general [39]. The reasoning in the previous paragraph demonstrates that they can be applied directly to Quasi-Bayesian inferences as well. Simulated annealing can guide a Gibbs sampler in generating samples of the posterior distribution; the sample with the highest probability defines the maximum [39].

This sampling approach offers a contrast between the interior-point methods advanced here and combinatorial optimization methods that search for the best combination of transparent variable values [5]. In combinatorial approaches, each iteration of the sampling procedure demands a complete cycle of standard Bayesian inference. Instead, by searching in the interior space of distributions, we can use the simulated annealing and Gibbs sampling simultaneously; the convergence of this process is a particular benefit of the probabilistic structure of graphical models [39] which is not exploited by purely combinatorial approaches [5].