\lecturetitle{23}{Evolutionary Trees and Web Indexing}{19 Nov 97}{Amar Chaudhary}


%%@
% INCLUDE AN OUTLINE OF THE CLASS TOPIC

\section*{Outline}
Today we have two student lectures.
\begin{itemize}
  \item Evolutionary Trees ({\it Andris Ambainis})
    \begin{enumerate}
    \item Parsimony
    \item Maximum-Likelihood
    \item Distance Methods
    \end{enumerate}
  \item Finding Authoritative Web Pages Using the Web's Link Structure ({\it David Wagner})
\end{itemize}

%%@
\section{Evolutionary Trees (Andris Ambainis)}

%%@
An evolutionary tree, or $phylogeny$, is a tree which describes the
ancestral relationships among a set of species.  The leaves of an
evolutionary tree correspond to the species being compared, and
internal nodes correspond to (possibly extinct) ancestral species.
Figure~\ref{fig1:23} shows a simple evolutionary tree for cats, dogs
and lynxes.  Domestic cats and lynxes share a feline ancestor distinct
from dogs, but all three species share a common ancestor as well.

\begin{figure}[phbt]

\setlength{\unitlength}{0.00083300in}
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(6012,3423)(1801,-4897)
\thicklines
\put(3976,-2686){\vector( 1,-2){960}}
\put(6376,-1486){\vector(-3,-1){2520}}
\put(3676,-2611){\vector(-2,-3){1269.231}}
\put(6526,-1561){\vector( 1,-2){1455}}
\put(6901,-4861){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}Dog (canis familiaris)}}}
\put(1801,-4861){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}Cat (felix domesticus)}}}
\put(4201,-4861){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}Lynx (lynx canadiensis)}}}
\end{picture}

\caption{An evolutionary tree for cats, dogs and lynxes}
\label{fig1:23}
\end{figure}

Scientists have been studying and creating phylogenies for over a
century.  Traditionally, morphological information and taxonomies were
used as data for creating evolutionary trees.  This process was often
guided more by intuition than a rigorous set of rules.  Today, more
systematic methods based on DNA sequencing are used.  DNA sequencing
provides a much larger body data which can be precisely analyzed.
Such data both demands and facilitates computer algorithms for
generating evolutionary trees.

Three algorithms for creating evolutionary trees will be discussed in
the following subsections.  In each case, we assume that the DNA
sequences have already been aligned using multiple alignment, and that
each position in the sequence is independent. 


\subsection{Parsimony}

In the parsimony method, the best tree is the one which explains the
data using the smallest number of evolutionary changes:

\begin{figure}[phbt]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(3087,1935)(1951,-2188)
\thicklines
\put(3601,-436){\vector( 3,-2){951.923}}
\put(4501,-1336){\vector(-3,-4){513}}
\put(4651,-1336){\vector( 2,-3){392.308}}
\put(2626,-1261){\vector(-2,-3){450}}
\put(2701,-1261){\vector( 1,-2){345}}
\put(2401,-1186){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}ACTA}}}
\put(1951,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}ACGT}}}
\put(2851,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}AGTA}}}
\put(3451,-436){\vector(-3,-2){813.462}}
\put(3751,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}CCAA}}}
\put(4876,-1636){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}0}}}
\put(4876,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}ACAA}}}
\put(4276,-1261){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}ACAA}}}
\put(3301,-361){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}ACAA}}}
\put(2251,-1636){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}2}}}
\put(2926,-1561){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}1}}}
\put(2926,-661){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}1}}}
\put(4051,-661){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}0}}}
\put(4351,-1711){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{bf}1}}}
\end{picture}}
\caption{A parsimony model for sequences ACGT, AGTA, CCAA and ACAA.
The numbers on the edges indicate the number of mutations between
nodes}
\label{fig2:23}
\end{figure}

Parsimony can be formulated as a Steiner Tree problem.
For example, given a set of $m$ aligned DNA sequence of length $n$,
consider the $n$ dimensional hypercube
$G=\{A,G,C,T\}^n$.  The weights on the edges of this graph
are the distance metric between the two endpoints.
Each DNA sequence is a vertex on this graph.
The  problem of parsimony is to find the minimum steiner tree with
these vertices as demand points.

\subsubsection*{Analysis}

If we are given the tree structure with the leaves assigned, we can
find a minimum-cost assignment for the internal nodes using dynamic
programming.  However, to find the tree that minimizes the number of
changes is an {\it NP}-hard problem.

We can approximate the solution, however, using local search
heuristics.  First, we find a ``good'' tree.  One way to do this is
using a greedy method: starting with the set of species, keep
connecting the two subtrees with the most closely-related roots until
there is only one tree.  We then apply rearrangement operations to the
tree and check if the rearranged tree has a lower cost.  Possible
rearrangement operations include:

\begin{itemize}
 \item Swap two subtrees that are children of neighboring nodes.
 \item Remove a subtree and attach it to some other node.
\end{itemize}

\noindent This process can be repeated a fixed number of times, or
until we find a local minimum.

\subsubsection*{Discussion}

Parsimony assumes that mutations are rare.  In fact, this may not the
case.  Indeed, it is possible using parsimony to construct
evolutionary trees that are ``positively misleading'' ({\em i.e.}, the
probability of getting the wrong tree approaches one as the amount of
data approaches infinity) \cite{xJ78}.

Despite being NP-hard and positively misleading, parsimony remains
popular among evolutionary biologists.  It has the advantage of being
simple, intuitive and appealing.  A recent study of Rice and Warnow
shows that on artificial data, parsimony performs quite well
\cite{xRW97}.

\subsection{Maximum-Likelihood}

The maximum-likelihood method generates an evolutionary tree from a
stochastic model of evolution.  The same model can be used to generate
trees for different sets of species.

To simplify this presentation, we first assume that DNA sequences are
binary ({\em i.e.}, sequences of 0's and 1's).

We will describe stochastic models called {\it Cavender-Farris trees}.
Such a tree has a probability associated with each edge as well as an
initial state probability associated with the root.  Given a tree with
root probability $p_R$ and probabilities $p_e$ for each edge $e$, each
position of the binary sequence is generated as follows:

\begin{enumerate}
 \item The root is labeled 1 with probability $p_R$ and 0 with
 probability $1-p_r$.

 \item Each labeled node broadcasts its label to all its children.

 \item When a label is broadcast over edge $e$, it gets reversed with
 probability $p_e$.
\end{enumerate}
Figure~\ref{fig3:23} shows an example of a Cavender-Farris
tree.  We generate a sequence for a node $X$ ({\em i.e.}, species $X$) by
repeatedly running the stochastic process and adding the label of $X$
to the sequence, as illustrated in figure~\ref{fig4:23}. This method
relies on the assumption that all positions in a DNA sequence are
independent.

\begin{figure}[pbht]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(2199,2910)(2239,-3163)
\thicklines
\put(2851,-1711){\vector(-1,-2){600}}
\put(3301,-2836){\makebox(6.6667,10.0000){\SetFigFont{10}{12}{rm}.}}
\put(3601,-436){\vector( 1,-3){825}}
\put(2926,-1711){\vector( 1,-3){375}}
\put(2251,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}A}}}
\put(3451,-436){\vector(-1,-2){600}}
\put(3226,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}B}}}
\put(4051,-1561){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0.01}}}
\put(4426,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}C}}}
\put(3451,-361){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0.5}}}
\put(2851,-1111){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0.2}}}
\put(2401,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0.4}}}
\put(3151,-2161){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0.01}}}
\end{picture}}
\caption{A stochastic model as a Cavender-Farris tree}
\label{fig3:23}
\end{figure}

\begin{figure}[phbt]
\centerline{
\setlength{\unitlength}{0.00053300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(11262,3285)(901,-3163)
\thicklines
\put(2251,-136){\vector( 1,-3){825}}
\put(1426,-1636){\vector(-1,-2){450}}
\put(1651,-1636){\vector( 1,-2){450}}
\put(901,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}A}}}
\put(2026,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}B}}}
\put(3076,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}C}}}
\put(6376,-136){\vector(-1,-2){600}}
\put(6526,-136){\vector( 1,-3){825}}
\put(5701,-1636){\vector(-1,-2){450}}
\put(5926,-1636){\vector( 1,-2){450}}
\put(5176,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}A}}}
\put(6301,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}B}}}
\put(7351,-3136){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}C}}}
\put(11176,-61){\vector(-1,-2){600}}
\put(11326,-61){\vector( 1,-3){825}}
\put(10501,-1561){\vector(-1,-2){450}}
\put(10726,-1561){\vector( 1,-2){450}}
\put(9976,-3061){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}A}}}
\put(2101,-136){\vector(-1,-2){600}}
\put(11101,-3061){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}B}}}
\put(9976,-1936){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}flip}}}
\put(12151,-3061){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}C}}}
\put(2101, 14){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0}}}
\put(1426,-1486){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0}}}
\put(901,-2686){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(2026,-2686){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0}}}
\put(3151,-2761){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0}}}
\put(6376, 14){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(5701,-1561){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(5176,-2686){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(6376,-2761){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(7351,-2761){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(9976,-2611){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}0}}}
\put(11176,-2611){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(12151,-2686){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(10501,-1486){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(11251, 14){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}1}}}
\put(901,-2011){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}flip}}}
\end{picture}
}
\caption{Generating sequences from the stochastic model in
figure~\protect\ref{fig3:23}. Edges over which the label has been
flipped are marked with ``flip.''}
\label{fig4:23}
\end{figure}

When using four states ({\em i.e.}, A,G,C,T), each edge is assigned a
probability matrix instead of a scalar probability $p_e$:

\[\left[ \begin{array}{llll}
p_{\rm AA} & p_{\rm AC} & p_{\rm AG} & p_{\rm AT} \\
p_{\rm CA} & p_{\rm CC} & p_{\rm CG} & p_{\rm CT} \\
p_{\rm GA} & p_{\rm GC} & p_{\rm GG} & p_{\rm GT} \\
p_{\rm TA} & p_{\rm TC} & p_{\rm TG} & p_{\rm TT} \end{array} \right] \]


\noindent where $p_{\rm XY}$ is the probability that X changes to Y.
Biologists often assume that some or all $p_{\rm XY}$ are related, or
even equal.

Given a stochastic model of evolution, the maximum-likelihood method
finds a tree $T$ that maximizes $p_{T}(S)$ where $S$ is the set of DNA
sequences and $p_{T}(S)$ is the probability that $T$ will generate all
the sequences in $S$.

This method is much more general than parsimony, which has a fixed
model of evolution that may be incorrect.  Assuming that the chosen
stochastic model of evolution is accurate, this method will give a
correct evolutionary tree for any set of species.  However, it is the
most computationally expensive of the methods discussed in this
lecture, and is rarely used on large trees.

\subsection{Distance methods}

In the distance method, we create a matrix $D$ consisting of
similarity measures between pairs of species in the set being studied.
Then, we find a tree that generates the distance matrix closest to
$D$.  This method is illustrated by the following example.

\subsubsection*{An example}

Take three species, $a$, $b$ and $c$ with the following DNA sequences:

\begin{tabbing}
$a$: \={ACAAAAACAA}\\
$b$:\>{ACTTGAACAT}\\
$c$:\>{TCTAAAACAT}\\
\end{tabbing}

\noindent Now generate a similarity matrix for $a$, $b$ and $c$:

\[ \left[ \begin{array}{ccc}
- & 0.4 & 0.3 \\
0.4 & - & 0.3 \\
0.3 & 0.3 & - \end{array} \right] \]

\begin{tabbing}
The \=similarity measure used is $d_{xy}=M/L$ where:\\
\>$M$ is the number of positions with synonymous characters.\\
\>$U$ is the number of positions with non-synonymous characters.\\
\>$G$ is the number of positions with gaps in one sequence and residue in another.\\
\>$W_G$ is the weight of gaps. (For this example $W_G=1$)\\
\>$L = M + U + W_{G}G$
\end{tabbing}

\noindent Transform the similarity matrix into a matrix with estimates
of ``evolutionary time'' between species.  We estimate the
evolutionary time $t_{xy}$ between two species $x$ and $y$ as
$t_{xy}={\ln(1-2d_{xy})}$.

\[ \left[ \begin{array}{ccc}
- & 1.62 & 0.93 \\
1.62 & - & 0.93 \\
0.93 & 0.93 & - \end{array} \right] \]

\noindent Now, find a tree with nodes $a$, $b$ and $c$ such that the
distance between each of these nodes is the same as the evolutionary
time between the corresponding species:

\begin{figure}[pbht]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(1662,2649)(826,-2488)
\thicklines
\put(1501,-1111){\line(-1,-2){525}}
\put(1576,-1111){\line( 3,-4){900}}
\put(1576, 89){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}c}}}
\put(1576, 14){\line( 0,-1){975}}
\put(826,-2386){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}a}}}
\put(2401,-2461){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{it}b}}}
\put(1726,-436){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}.12}}}
\put(2026,-1486){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}.81}}}
\put(1051,-1561){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}.81}}}
\end{picture}
}
\caption{An evolutionary that fits the time matrix for species $a$,
$b$ and $c$.}
\label{fig5:23}
\end{figure}

\noindent Note that the tree in figure~\ref{fig5:23} is undirected.


\subsubsection*{Analysis}

If the data precisely fits some tree, then this method is relatively
simple.  However, real data have random errors.  If no tree exactly
fits the data, finding the tree that is closest to a distance matrix
is {\it NP}-complete under several definitions of ``closeness.''  In
practice, the following heuristic, called {\it nearest-neighbor
joining} \cite{xSN87} is often used:

\begin{itemize}
 \item Introduce closeness measure $d'(x,y) = (n-2)d(x,y) - \sum_xd(x,z) - \sum_yd(y,z)$
 \item Repeat until only one node remains:
 \begin{enumerate}
  \item Search for $x$ and $y$ that minimize $d'(x,y)$.
  \item Replace $x$ and $y$ with a new node $xy$.
  \item Calculate $d(xy,z) = (d(x,z)+d(y,z)-d(x,y))/2$
 \end{enumerate}
\end{itemize}

\noindent The evolutionary tree will the tree where each pair of nodes
$x$ and $y$ is connected its joint $xy$:

\begin{figure}[pbht]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(1446,1175)(725,-1286)
\thicklines
\put(901,-1111){\circle{336}}
\put(1951,-661){\circle{424}}
\put(1051,-1111){\vector( 2, 1){750}}
\put(901,-286){\circle{336}}
\put(1051,-286){\line( 3,-1){675}}
\put(1726,-511){\vector( 1, 0){150}}
\put(1876,-736){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}xy}}}
\put(901,-361){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}x}}}
\put(901,-1186){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}y}}}
\end{picture}
}

\caption{A branch of the evolutionary tree joining nearest neighbors
$x$ and $y$.}

\label{fig6:23}
\end{figure}


\section{Finding authoritative web pages using link structure (David Wagner)}

Today's web search engines do keyword searching.  This technique is
very powerful, but it handles broad queries quite poorly.  The
quantity of hits is often very high, and the quality of these hits
varies widely.

We want to be able to identify authoritative sources for broad
queries.  This lecture presents an algorithm which:

\begin{enumerate}
 \item Views the web as a directed graph.
 \item Picks a relevant subgraph using standard search techniques.
 \item Analyzes the resulting link structure.
\end{enumerate}

Why analyze the link structure?  Take two pages $p$ and $q$ such that
$p$ has a link to $q$.  Through this link, the creator of page $p$ has
in some sense conferred authority on $q$.  For example, the
``Algorithms in the Real World'' class page contains a large number of
links to related pages.  These linked pages can be considered
authoritative sources on algorithms in the real world.

\subsection{A naive approach using adjacency}

The ``authoritativeness'' of $p$ can be measured using the {\it
in-degree} of $p$, or the number of pages which link to $p$.  More
precisely, we want the in-degree of $p$ in a subgraph induced by the
keyword search for the query.  Such a subgraph can be generated by
taking the top 200 pages returned by a standard search engine (like
AltaVista or HotBot) as a root set, and then adding all ``adjacent''
pages ({\em i.e.}, pages with links into and out of the root set).

This approach does not work very well in practice.  For example, a
search on the word ``java'' will probably return pages on the
programming language, the island in Indonesia and the popular
caffeinated beverage.  It is likely that the user only wanted one of
these topics.

However, an analysis of the link structure in this query result will
likely reveal three subgraphs, separated roughly by topic and with few
interconnecting links between them.  This property leads to an
improved search algorithm.

\subsection{An algorithm for finding communities within topics}
\index{Authoritative Pages: Kleinberg}

{\it Hubs} are pages that point to multiple relevant authoritative
pages.  The best known example of a hub is Yahoo.  An {\it authority}
is a page that is pointed to by multiple relevant hubs.  For example,
the JavaSoft home page is an authority, since is linked by most hubs
that catalogue Java links.

We want to find {\it communities}, or sets of hubs $H$ and authorities
$A$ such that $H+A$ resembles a dense bipartite graph.

\begin{figure}[pbht]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(2152,2900)(350,-2411)
\thicklines
\put(601,-586){\circle{336}}
\put(676,-1561){\circle{336}}
\put(751,-2236){\circle{336}}
\put(2251,-1036){\circle{336}}
\put(2176,-1561){\circle{336}}
\put(2326,-211){\circle{336}}
\put(676,239){\vector( 4,-1){1500}}
\put(676,239){\vector( 4,-3){1500}}
\put(676,239){\makebox(6.6667,10.0000){\SetFigFont{10}{12}{rm}.}}
\put(526,314){\circle{336}}
\put(676,239){\vector( 3,-4){1350}}
\put(901,-2161){\vector( 2, 3){1200}}
\put(751,-511){\makebox(6.6667,10.0000){\SetFigFont{10}{12}{rm}.}}
\put(751,-586){\vector( 3, 1){1350}}
\put(751,-586){\vector( 3,-1){1350}}
\put(751,-586){\vector( 4,-3){1200}}
\put(826,-1486){\vector( 1, 1){1275}}
\put(901,-1486){\vector( 3, 1){1125}}
\put(901,-1561){\vector( 1, 0){1050}}
\put(901,-2161){\vector( 2, 1){1050}}
\end{picture}
}

\caption{A subgraph representing a community.  Hubs (on the left)
point to authorities (on the right).}

\label{fig7:23}
\end{figure}

We will now define an iterative algorithm for finding
communities. Page $p$ is assigned weights $x[p]$ and $y[p]$ that
represent authority weight and hub weight, respectively.  The hub
weight $x[p]$ is the sum of authority weights of its outgoing links.
Likewise, $y[p]$ is the sum of the hub weights of pages that point to
$p$. Figure~\ref{fig8:23} illustrates how $x[p]$ and $y[p]$ are
calculated.

\begin{figure}[phbt]
\centerline{
\setlength{\unitlength}{0.00083300in}%
%
\begingroup\makeatletter\ifx\SetFigFont\undefined
% extract first six characters in \fmtname
\def\x#1#2#3#4#5#6#7\relax{\def\x{#1#2#3#4#5#6}}%
\expandafter\x\fmtname xxxxxx\relax \def\y{splain}%
\ifx\x\y   % LaTeX or SliTeX?
\gdef\SetFigFont#1#2#3{%
  \ifnum #1<17\tiny\else \ifnum #1<20\small\else
  \ifnum #1<24\normalsize\else \ifnum #1<29\large\else
  \ifnum #1<34\Large\else \ifnum #1<41\LARGE\else
     \huge\fi\fi\fi\fi\fi\fi
  \csname #3\endcsname}%
\else
\gdef\SetFigFont#1#2#3{\begingroup
  \count@#1\relax \ifnum 25<\count@\count@25\fi
  \def\x{\endgroup\@setsize\SetFigFont{#2pt}}%
  \expandafter\x
    \csname \romannumeral\the\count@ pt\expandafter\endcsname
    \csname @\romannumeral\the\count@ pt\endcsname
  \csname #3\endcsname}%
\fi
\fi\endgroup
\begin{picture}(3075,3136)(1,-2647)
\thicklines
\put(376,-286){\circle{336}}
\put(376,-886){\circle{336}}
\put(2251,-361){\circle{336}}
\put(2176,-1411){\circle{336}}
\put(2176,-1936){\circle{336}}
\put(2176,-2461){\circle{336}}
\put(301,-1936){\circle{336}}
\put(676,239){\vector( 3,-1){1417.500}}
\put(601,-286){\vector( 1, 0){1500}}
\put(601,-811){\vector( 4, 1){1500}}
\put(451,-1936){\vector( 3, 1){1575}}
\put(451,314){\circle{336}}
\put(451,-1936){\vector( 1, 0){1575}}
\put(3076,-1936){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}$y[p] = \sum x[q_i]$}}}
\put(451,-1936){\vector( 3,-1){1575}}
\put( 76,314){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q1}}}
\put(  1,-361){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q2}}}
\put(  1,-961){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q3}}}
\put(2476,-1486){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q1}}}
\put(2476,-2011){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q2}}}
\put(2476,-2611){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}q3}}}
\put(301,-2011){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}p}}}
\put(2251,-436){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}p}}}
\put(3076,-436){\makebox(0,0)[lb]{\smash{\SetFigFont{12}{14.4}{rm}$x[p] = \sum y[q_	i]$}}}
\end{picture}
}
\caption{Calculating the hub weight and authority weight of a page $p$}
\label{fig8:23}
\end{figure}

The algorithm can be implemented using linear algebra.  Let $A$ be the
adjacency matrix of the graph induced by the keyword search.  $A_{pq}$
is 1 if $p$ has a link to $q$ and 0 otherwise.  At each iteration, let
$x = A^Ty$ and $y = Ax$.  Normalize $x$ and $y$, and repeat the
process until $x$ and $y$ converge.  From linear algebra, we know that
$x$ and $y$ will eventually converge to the principal eigenvectors of
$A^TA$ and $AA^T$.

Each eigenvector corresponds to one community.  The principal
eigenvector corresponds to the most prominent community, but other
eigenvectors can be used to examine other communities in the topic.
It is interesting to note the similarities between the linear algebra
approach to finding communities and the use of SVD in latent semantic
indexing as described in the previous lecture.  Recall that in finding
the $SVD(A) = U \Sigma V^T$ that the columns of $U$ are the normalized
eigenvectors of $AA^T$ and the columns of $V$ are the normalized
eigenvectors of $A^TA$.   Using $U_1$ and $V_1$ gives us the
principal eigenvectors (the ones with the highest eigenvalue).
In fact, the iterative approach mentioned above (often called
repeated squaring) is just one of the many approaches to solve
for the first eigenvectors for the SVD.

Here are some examples of real queries using this algorithm:

\begin{itemize}

 \item The principal eigenvector of the query ``java'' includes
 JavaSoft, the {\tt comp.lang.java} FAQ, etc.

 \item The query ``net censorship'' includes the sites for EFF, CDT,
 VTW and ACLU, all of which are organizations that play a prominent
 role in free speech issues on the net.

 \item The query ``abortion'' returns pro-life pages as one
 eigenvector and pro-choice pages as another.  Each side of the issue
 tends to form a bipartite graph with very few cross-links to the
 opposition.

\end{itemize}

\noindent This method works surprisingly well for broad queries, and
shows promise for some other search tasks, such as ``similar page''
queries ({\em i.e.} search for pages with information similar to a given
page).  However, for very specific queries, the authoritative search
does not work very well.  Rather, it tends to find authorities for
generalizations of the query topic.  It might be possible to use
lexical ranking function to keep the authoritative search oriented,
but this is left open for future work.

\subsection*{For further reading}

More details on the authoritative searching technique can be found
elsewhere. \cite{xK97}.

\begin{thebibliography}{widest-label}

\bibitem[J78]{xJ78} J. Felsenstein, ``Cases in which parsimony or
compatibility methods will be positively misleading.'' {\it Systematic
Zoology}, 27:401-410, 1978.

\bibitem[K97]{xK97} J. Kleinberg. ``Authoritative Sources in a
Hyperlinked Environment.''  IBM Research Report RJ 10076, May 1997.
This paper will also appear in the {\t Proceedings of the ACM SIAM
Symposium on Discrete Algorithms}, 1998.

\bibitem[RW97]{xRW97} Kenneth Rice and Tandy Warnow, ``Parsimony is
Hard to Beat.'' {\it COCOON97}.

\bibitem[SN87]{xSN87} N. Saifou, M. Nei.  ``The neighbor-joining
method: a new method for unconstructing phylogenic trees.''  {\it
Mol.Biol.Evol.}, 4:406-425, 1987.

\end{thebibliography}
