\section{Evaluation}
\label{perf}

  To evaluate the MacFS library, We addressed the issues of the portability
of the code, the versatility of the m programming interface, and
the functionality and performance of the implementation.  

\subsection{Portability and Versatility}

  We developed the MacFS library on two very different machines, a
DECstation 5000/120 and an Intel 486 system.  The significant
differences between these machines demonstrates the portability of the
code.  Further, we implemented both a user level library and a Unix
VFS on top of the m programming interface of the MacFS library,
demonstrating the versatility of this interface.  Other implementors
have used MacFS to build an OS/2 IFS driver and for accessing Macintosh 
media from a professional digital audio recorder.


\subsection{Functionality}

  We used the Macintosh Finder and the Macintosh Norton Utilities 2.0
to test that disks modified by our user-level and VFS-level MacFS
implementations remain Macintosh compatible.  Although we are able to
copy any type of file, all new files created have the Finder Info of
text files, unless modified using macsetfinderinfo().  

\subsection{Performance}

  We evaluated the performance of our user level library, the native
Macintosh file system, and the Unix FFS using five simple benchmarks:

\begin{itemize}
\item Create: Create a large number of files.

\item Delete: Delete a large number of files.

\item Write: Create, write, and close one 5,000,000-byte file with
request sizes of 500, 5,000, and 50,000 bytes.

\item Overwrite: Open, overwrite and close one 5,000,000-byte file 
using the same range of block sizes.

\item Read: Read one 5,000,000-byte file using the same range of block sizes.

\end{itemize}

  The the difference between the Write and Overwrite benchmarks is
that Overwrite does not include the cost of growing the file.  All
reads and writes were sequential.  The benchmarks were run on a 240 MB
Hitachi DK312C disk on a DECstation 5000/120 (24 MHz MIPS R3000) and
on a Macintosh 145B (25 MHz MC68030.)  On the Macintosh, the
benchmarks were run using the native OS and file system, APS SCSI
driver 2.7, and the Think C 5.0 Unix I/O emulation libraries.  In all
cases, the disk was formatted with a 4K logical block size.

  Figure \ref{Performance} compares the performance of the three file
systems.  Note that the MacFS library can create and delete files
three times as fast as the native Macintosh file system.  There are
three explanations for this.  The first is that the Macintosh may have
to send refresh messages to open Finder windows and dialog boxes on
file creation or deletion.  The second is that the Macintosh may
immediately update the on-disk mount record and free-space bitmap,
something that MacFS defers.  Finally, initializing Finder Information
may be more complex on the Macintosh.  The Unix FFS is much faster
than either MHFS or MacFS due to the greater simplicity of FFS
metadata.

  In the Write, Overwrite, and Read tests, the MacFS library is
consistently slower than the Unix FFS, but faster than the native
Macintosh file system for all but the largest request size.  At 50,000
bytes per request the native Macintosh is even faster than the Unix
FFS.  We suspect that with larger request sizes the Macintosh is
better able to amortize the cost of extent lookup, or that a different
extent lookup algorithm comes into play with very large request sizes.
Noting that MacFS read-write speeds do not increase monotonically with
request size and that read performance seems to peak around 5,000
bytes per request, the latter suspicion is probably the case.

  The performance of our VFS implementation is not sufficient to
warrant inclusion in the table.  Because it was intended mainly as a
proof-of-concept of the versatility of the m interface, we did not
tune its performance.


\begin{figure}
\begin{center}
\begin{tabular}[h]{|l|r|r||r|r|r||r|r|r||r|r|r|}
\hline
& \multicolumn{1}{c|}{}
& \multicolumn{1}{c||}{}
& \multicolumn{3}{c||}{500 bytes/req}
& \multicolumn{3}{c||}{5,000 bytes/req}
& \multicolumn{3}{c|}{50,000 bytes/req} \\
\hline Platform & Create & Delete
& Write & Over. & Read
& Write & Over. & Read
& Write & Over. & Read \\
\hline Macintosh & 7.5 & 7.5
& 26.5  & 27.0  & 28.7
& 118.2  & 119.2  & 124.2
& 527.0 & 532.9  & 548.9 \\
\hline MacFS & 24.6 & 23.2
 & 83.0  & 71.1 & 173.6
 & 147.8  & 182.4 & 303.8
 & 211.8  & 169.7 & 303.2 \\
\hline Unix FFS & 111.5 & 571.7
& 265.7  & 150.0 & 254.7
& 373.7  & 139.7 & 406.2
& 410.1  & 388.8  & 398.7  \\
\hline  & files/s & files/s
 & KB/s  & KB/s & KB/s
 & KB/s  & KB/s & KB/s
 & KB/s & KB/s & KB/s \\
\hline
\end{tabular}
\end{center}
\caption{Performance Comparison between the native Macintosh file system, 
the MacFS user library, and the Unix FFS}
\label{Performance}
\end{figure}

%
% This is the data for the rodime disk
%
%%\begin{table}
%\begin{center}
%\begin{tabular}[h]{|l|r||r|r|r||r|r|r||r|r|r|}
%\hline
%& \multicolumn{1}{c||}{}
%& \multicolumn{3}{c||}{500 bytes/request}
%& \multicolumn{3}{c||}{5000 bytes/request}
%& \multicolumn{3}{c|}{50000 bytes/request} \\
%\hline Platform & Create
%& Write & Over. & Read
%& Write & Over. & Read
%& Write & Over. & Read \\
%\hline Macintosh & 7.5?
%& 27.2  & 27.6  & 150.2 
%& 172.1  & 172.9  & 443.1 
%& 546.4  & 561.8  & 842.7  \\
%\hline Unix FFS & 230?
%& 254.4  & - & -
%& 442.7  & - & -
%& 344.7  & 355.7  & 292.0  \\
%\hline Unix MacFS & ?
%& 21.1  & - & -
%& 24.0  & - & -
%& 28.9  & 27.0  & 85.2  \\
%\hline MacFS User level & 46.7?
% & 83.4  & - & -
% & 203.0  & - & -
% & 322.9  & - & - \\
%\hline  & files/sec
% & KB/s  & KB/s & KB/s
% & KB/s  & KB/s & KB/s
% & KB/s & KB/s & KB/s \\
%\hline
%\end{tabular}
%\end{center}
%\caption{MacFS Performance}
%\label{Performance}
%\end{table}