\subsection{Ranks}

Throughout this paper, we use {\em rank} to compare servers'
performance.  In this section we explain how ranks are computed and
give some insight into what differences in rank mean.  A ranking of
servers is computed for each data set (Mars, News, or Apache) for each
group of fetches at each client.  Recall that after each group of
fetches, a client has performance data for each web server.  For each
document, we can order the servers by their fetch times from lowest to
highest, discarding those servers whose fetches failed.  A server's
rank is merely its place in this order.  The server which comes first
in the order has the highest rank (0), the server which comes next has
a rank of 1, and so on.  In our terminology, lower ranks correspond to
better server performance.  In summary, each successful group of
fetches generates one set of ranks for each of the 11 documents: 5
sets for Mars documents, 5 for Apache documents, and one for the News
document.

There is some inaccuracy in our method of ranking servers: The tacit
assumption in computing ranks is that the fetch times being compared
were generated under identical conditions.  As we have discussed in
Section~\ref{section:limitations}, this is not possible, but we
believe that network conditions do not change a significant amount
between the first and last fetch of a document from a group of
servers.

\begin{figure}[t]
\begin{center}
\epsfig{file=mars-4-1.eps,width=3in}
\end{center}
\caption{Average amount of separation between rank values for Mars
servers, document 4, all clients aggregated.}
\label{mars-rank-separation}
\end{figure}

Ranks are not significant performance indicators by themselves.  Ranks
will not say whether or not the difference in performance between
servers is negligible.  But in the data that we collected, we have
found a very strong link between noticeable differences in performance
and differences in rank.

Figure~\ref{mars-rank-separation} plots the normalized, average
increase in transfer time vs. server rank for document 4 of the Mars
data set.  It was produced by averaging the differences of all pairs
of servers with ranks $i$ and $i-1$ in each group.  The graph shows a
definite rise in transfer time as rank increases.  For example, we see
that on average, a server with a rank of 4 has twice the transfer time
of a server with a rank of 0.  Further, the server with the largest
rank (17) takes more than 30 times as long to transfer a document as
the best server, and it takes more than 3 times as long to deliver a
document as a server with a rank of 14.

The primary point of Figure~\ref{mars-rank-separation} is that rank
changes usually correspond to noticeable performance changes for
document 4 of the Mars set.  All other documents from Mars, Apache,
and News produced similar graphs, though the Apache and News data
tended to have much larger differences in performance.  This gives us
confidence that ranks are a reasonable way to talk about the relative
performance of servers.

Figure~\ref{mars-rank-separation} also answers our first question
about whether performance varies across mirror servers.  A factor of
30 separates the transfer times of the best and worst servers.  Even
the top 10 servers' performance varies by a factor of 5.  We noticed
the same or more exaggerated ranges of performance for other documents
and sites.  Performance definitely varies widely across mirrors of a
site.