\documentclass[11pt]{article}
\usepackage{latexsym}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{epsfig}
\usepackage[tight]{subfigure}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{pstricks,pst-node}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {#1} \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[5]{\handout{#1}{#2}{#3}{#4}{#5}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex

\DeclareMathOperator*{\argmax}{arg\,max}

%\renewcommand{\baselinestretch}{1.25}

\begin{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Document header: change the lecture number, date, scribe name, and
% lecture topic.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\lecture{Statistical Techniques in Robotics (16-831, F10)}
{Lecture\#08 (Thursday September 16)}
{Lecturer: Drew Bagnell}
{Scribe:Debadeepta Dey \footnotemark}
{Gibbs Fields: Inference and Relation to Bayes Networks}

% Be sure to acknowledge all previous scribes.
\footnotetext{Some content adapted from previous scribes: Byron Boots}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Document body: Scribe your notes here.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Inference on Gibbs Fields}

Once we have 'learnt' a Gibbs Field or a Markov Random
Field (both of them are equivalent according to the 
Hammersly-Clifford theorem) the main question is how 
to do inference on the learnt graphical model.
There are two major types of queries that are useful
in general.

\begin{enumerate}
 \item Marginals
 \item Most probable assignment given some observed nodes
\end{enumerate}

\subsection{Marginals}
\begin{equation}
  P(X_{i}|observedX's)
\end{equation}

The naive way of doing marginals is to sum over all possible 
combination of assignments to the unobserved variables. 

Assuming that we have a graph whose clique potentials are
Gaussians we can write the marginal in the conventional way
as:

\begin{equation}
 p(\vec{X}|observedX's) = (1/Z)*exp(-\sum_{i}\sum_{j \in N_{i}}f(X_{i}, X_{j}))
\end{equation}

Here $N_{i}$ are the neighbors of the $i^{th}$ node. If the random variables
in the node are binary and there are $n$ such nodes and $k$ of these nodes 
are observed then the number of elements is of the order of $2^{n-k}$. This is 
generally intractable as the number of nodes in a graph can easily be of the 
order of a few thousands or more.

We will explore 2 methods of doing this: 

\begin{enumerate}
 \item Importance Sampling
 \item Gibbs Sampling
\end{enumerate}

\subsubsection{Importance Sampling}
Generate samples from 
\begin{equation}
 P(X) = \prod_{i}p(X_{i})
\end{equation}
while clamping nodes observed to the observed values. This means that 
we are not sampling the original distribution and hence we must weight 
each of the samples by the probability of the particular joint assignment.

The marginal is then obtained from all the weighted samples by the ratio of the 
sum of the weights of the samples which satisfy the query divided to the total 
weight of all samples.

\begin{equation}
 p(\vec{X}|observedX's) = \frac{N_c}{N}
\end{equation}

where $N_{c}$ = sum of weights of samples which satisfy the query and $N$ is the total
weight of all samples.
 
\subsubsection{Gibbs Sampling}

The best way to explain Gibbs Sampling is to go over an example of how to do it and 
then try to get the intuition behind the procedure.

\begin{figure}[htbp]
  \label{lattice}
 \begin{center}
  \includegraphics[scale=0.3]{lattice.eps}
  \caption{Example lattice graph with a single node observed as $-1$. This 
  kind of single layer graph is also called an 'Ising' model.}
 \end{center}
\end{figure}

Refer to Fig.\ref{lattice}. It is a simple lattice structure also commonly called as the 'Ising' model.
Here the node in red has been observed to be the value $-1$. Given this observation we want to know what
the values of the rest of the nodes in the graph. 

The pseudo-code for the Gibbs sampling algorithm is given in Algorithm.\ref{gibbs_code}.
In line $1$ you can randomly assign any value to all the unobserved nodes while fixing
the observed nodes to the values observed . In lines
$3-5$ you move over the graph fixing the value of every $i^{th}$ node under consideration
by sampling the conditional distribution for that particular node which is easy to do
as the node under consideration is independent of the rest of the graph
given its Markov blanket which in the case of Gibbs Fields are just its neighbors.
This takes the form $P(x_{i}|N(x_{i})) = \frac{1}{Z}exp(\sum_{i,j \in N}f(x_{i}, x_{j}))$ where
$N$ is the set of neighbors of $x_{i}$. Please note that the updated value of $x_{i}$ is
used as soon as it is computed while computing the value of its neighbor. This fact is
noted in Algorithm.\ref{gibbs_code} in line $4$ where some of the nodes have 
$t+1$ in the index while others have $t$. Once you have gone over the whole 
graph once this  completes the inner loop and you start over with the current updated 
state of the graph. All through the procedure the observed nodes remain fixed.

\begin{algorithm}
\caption{Pseudo-code for the Gibbs sampling method on Gibbs Fields}
\label{gibbs_code}
\begin{algorithmic}[1]
 \STATE $X^{(0)} := <x_{1}^{0},...., x_{k}^{0}>$
 \FOR{$t = 1$ to $T$}
  \FOR{$i = 1$ to $K$}
    \STATE $x_{i}^{(t+1)} = P(x_{i}| x_{1}^{(t+1)}, ... , x_{i-1}^{t+1}, x_{i+1}^{t}, ... , x_{k}^{t})$
  \ENDFOR
  \ENDFOR
\end{algorithmic}
\end{algorithm}

\begin{figure}[htbp]
  \label{lattice_sample}
 \begin{center}
  \includegraphics[scale=0.3]{lattice_sample.eps}
  \caption{For the node under consideration (yellow), its
value is determined by the conditional distribution
over all its neighboring nodes shown in the figure. Then 
this process is repeated over all the nodes in turn while
keeping the observed nodes fixed to observed values.}
 \end{center}
\end{figure}


\paragraph{Intuition}
The Gibbs sampling procedure is an instance of the Metropolis-Hastings algorithm 
which is a type of Markov Chain Monte Carlo method. It is a Markov Chain method since
it is generating current samples depending only the last state of the entire graph.
It is a Monte Carlo method since the sampling is a probabilistic process. 

The idea is that by repeatedly sampling in the manner outlined the algorithm will 
converge to the actual distribution of the graph. The actual distribution of the 
graph can be thought of as the stationary distribution underlying the markov chain
we are generating in our sampling procedure. 

The reader is directed to the read the first part of the excellent tutorial \cite{gibbs_tutorial}
for more intuition and the second part for all the related mathematical details.

\paragraph{Gibbs Sampling Problems}
\subparagraph{Critical Phenomena}
This is also sometimes referred to as 'Critical Slowing Down'. 
This happens when one node starts determining the value of another
node really far away. This can start happening when the sampling
has gone through many iterations but there are regions of high probablity and 
low probability now in the graph. These regions may be very highly correlated and hence
will cause the sampling to produce series of values which are similar.

In general the stronger the connections between nodes the slower Gibbs 
sampling will run to converge in distribution.

Another good reference which goes into great detail on the topic is
\cite{mcmc_practice}. 

\subsection{ Most probable assignment given some observed nodes}
\begin{equation}
  \argmax_{\vec{X}}P(\vec{X}| observed X's)
\end{equation}

We will deal with this later.

\section{Gibbs Fields and Bayesian Networks}
You can go from Bayesian Networks to Gibbs Fields but not
vice versa. Even when you convert a Bayesian Network to 
a Gibbs field you are going to lose information as will be illustrated
in the example below.

You can not necessarily convert a Bayes' net into a Gibbs field. 
For example, consider the Bayes' net in Figure \ref{fig:bayesnet}. 
If you remove the arrows (Figure \ref{fig:convert1}), 
then the graph is \emph{not} equivalent. In particular observing 
node $B$ causes nodes $A$ and $C$ to become independent. 
This is the opposite of what the original graph represented. 
Instead, we need to \emph{moralize} the graph.  Whenever there 
are two parents that are not connected (married), we connect them. 
Thus, Figure \ref{fig:convert2} shows the correct representation of 
the original Bayes net. Note that during this conversion we actually
 lost information, namely that $A$ and $C$ are marginally independent.


\begin{figure}[h]
\begin{center}
\begin{pspicture}(-1,-1)(5,5)
\psset{arrowscale=2} 
\rput(0,4){\circlenode{a}{\makebox[1cm]{$A$}}}
\rput(2,0){\circlenode{d}{\makebox[1cm]{$D$}}}
\rput(2,2){\circlenode{b}{\makebox[1cm]{$B$}}}
\rput(4,4){\circlenode{c}{\makebox[1cm]{$C$}}}
\ncline[arrows=->]{a}{b}
\ncline[arrows=->]{c}{b}
\ncline[arrows=->]{b}{d}
\end{pspicture}
\end{center}
\caption{Bayes' Net}
\label{fig:bayesnet}
\end{figure}

\begin{figure}[h]
\begin{center}
\begin{pspicture}(-1,-1)(5,5)
\psset{arrowscale=2} 
\rput(0,4){\circlenode{a}{\makebox[1cm]{$A$}}}
\rput(2,0){\circlenode{d}{\makebox[1cm]{$D$}}}
\rput(2,2){\circlenode{b}{\makebox[1cm]{$B$}}}
\rput(4,4){\circlenode{c}{\makebox[1cm]{$C$}}}
\ncline[arrows=-]{a}{b}
\ncline[arrows=-]{c}{b}
\ncline[arrows=-]{b}{d}
\end{pspicture}
\end{center}
\caption{INCORRECT}
\label{fig:convert1}
\end{figure}

\begin{figure}[h]
\begin{center}
\begin{pspicture}(-1,-1)(5,5)
\psset{arrowscale=2} 
\rput(0,4){\circlenode{a}{\makebox[1cm]{$A$}}}
\rput(2,0){\circlenode{d}{\makebox[1cm]{$D$}}}
\rput(2,2){\circlenode{b}{\makebox[1cm]{$B$}}}
\rput(4,4){\circlenode{c}{\makebox[1cm]{$C$}}}
\ncline[arrows=-]{a}{b}
\ncline[arrows=-]{c}{b}
\ncline[arrows=-]{a}{c}
\ncline[arrows=-]{b}{d}
\end{pspicture}
\end{center}
\caption{CORRECT: Moralizing}
\label{fig:convert2}
\end{figure}

\subsection{Gibbs Sampling of Bayesian Networks}

\begin{thebibliography}{5}

\bibitem{gibbs_tutorial}Philip Resnik and Eric Hardisty, \textsl{Gibbs Sampling for the Uninitiated}, 
UMIACS-TR-2010-04, (2010).

\bibitem{mcmc_practice}Gilks, W.R. and Richardson, S. and Spiegelhalter, D.J., \textsl{Markov chain Monte Carlo in practice}, Chapman \& Hall/CRC, (1996).
  
\end{thebibliography}


\end{document}
