%% LyX 1.6.1 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[english]{spie}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}

\usepackage{babel}

\begin{document}

\title{Class Notes Summary on Sep 16}


\author{by Andres Rodriguez}

\maketitle

\section{Review from previous lecture}

Note on Bayes Nets: \[
P(A,B)=P(A)P(B|A)=P(B)P(B|A)=\phi(A,B)/z\]


This means that $A\rightarrow B$ is equivalent to $B\rightarrow A$
and $A--B$.

However, Bayes nets were originally made to encode causality. It is
much easier for a doctor to tell the symptoms given a disease, than
to tell the disease given the symptoms (since many diseases shared
the same symptoms).


\section{On-line learning}

This area is concerned with making decisions from limited information.
We begin by studying the problem of {}``predicting from expert advice.''
(Keep in mind that the word {}``expert'' may be deceiving since
many of these {}``experts'' may in reality have no clue.)

Let's say each day we have the same $n$ experts that predict yes
(1) or no (0) on whether is going to rain. After they make their prediction,
the algorithm makes its own prediction, and then finds out if it actually
it rains. Our goal is do perform nearly as well as the best expert
so far (being competitive with respect to the best single expert).


\subsection{Weighted Majority Algorithm}

This algorithm maintains a list of weights $w_{1},...,w_{n}$ (one
for each expert $x_{1},...,x_{n}$), and predicts based on a weighted
majority vote, penalizing mistakes by multiplying their weight by
half.


\paragraph{Algorithm}
\begin{enumerate}
\item Set all the weights to 1.
\item Predict 1 (rain) if $\sum_{x_{i}=1}w_{i}\geq\sum_{x_{i}=0}w_{i}$,
and 0 otherwise.
\item Penalize experts that are wrong: for all $i$ s.t. $x_{i}$ made a
mistake, $w_{i}^{t+1}\leftarrow\frac{1}{2}w_{i}^{t}$.
\item Goto 2
\end{enumerate}

\paragraph{Analysis of Algorithm}

The sum of the weights is $w\leq\left(\frac{3}{4}\right)^{m}n=\left(\frac{4}{3}\right)^{-m}n$,
where $m$ is the number of mistakes. The weight of the best expert
$w_{i}^{*}=\left(\frac{1}{2}\right)^{m^{*}}=2^{-m^{*}}\leq w.$ Therefore,\[
2^{-m^{*}}\leq w\leq\left(\frac{4}{3}\right)^{-m}n.\]


Taking the $\log_{2}$ and solving for $m$ gives,\[
m\leq\frac{m^{*}+\log_{2}n}{\log\frac{4}{3}}=2.41(m^{*}+\log_{2}n)\]



\subsection{Randomized Weighted Majority Algorithm}

In this algorithm, we view the weights as probabilities, and predict
each outcome with probability to its weight. We also penalized each
mistake by $\beta$ instead of just half.


\paragraph{Algorithm}
\begin{enumerate}
\item Set all the weights to 1.
\item Choose expert $i$ in proportion to $w_{i}$.
\item Penalize experts that are wrong: for all $i$ s.t. $x_{i}$ made a
mistake, $w_{i}^{t+1}\leftarrow\beta w_{i}^{t}.$
\item Goto 2.
\end{enumerate}

\paragraph{Analysis}

The bound is \[
E[m]\leq\frac{m*\ln(1/\beta)+\ln(n)}{1-\beta}.\]


We want $\beta$ to be small if we only have a few time steps available
and vice-verse.


\subsection{General Weighted Majority Algorithm}


\paragraph{Algorithm}
\begin{enumerate}
\item Set all the weights to 1.
\item Predict expert $i$ in proportion to $w_{i}$.
\item Receive the correct value $y_{t}$.
\item Adjust weights s.t. $w_{i}^{t+1}\rightarrow w_{i}^{t}\exp^{-\epsilon l(i,y_{t})}\forall i$,
where $l$ is the lost function, and $\epsilon$ is the penalizer.
\item Goto 2.
\end{enumerate}

\paragraph{Analysis}

The bound is\[
E[R]\leq\epsilon\sum l(i^{*})+\frac{1}{\epsilon}\ln(n),\]


where $R$ is the regret. If $\epsilon$ is large, we learn fast.
If $\epsilon$ is small, we learn slow but we'll have little regret.


\section{Intro to next lecture}

In a convex set, the line between any two points in the set is also
in the set. A convex function can be defined s.t.\[
f(ax_{1}+bx_{2})\leq af(x_{1})+bf(x_{2}),\]


where $a+b=1$, and $a,b\geq0$.
\end{document}
