\section{Conclusions}
\label{conc-sect}

In this paper, we have examined a phased AAPC algorithm and the
architectural requirements for its efficient execution.  Our prototype
implementation on iWarp shows that the phased AAPC architecture is
practical and efficient. The examination of other popular network
architectures showed that our AAPC support requires minimal additional
hardware and is compatible with most wormhole routers.

The iWarp implementation reaches an aggregate network bandwidth of
over 2 GB/s for balanced AAPC.  This is a factor of five faster than a
conventional message passing implementation on the same hardware.

The main reason for the performance improvement is the synchronizing
switch.  For optimal AAPC, the phases must be separated and a
contention-free schedule must be maintained.  Our synchronizing switch
design waits for the tails of all messages to propagate through a node
before sending the next phase's messages. It achieves this by enforcing
a local constraint rather than invoking a global barrier
synchronization operation. Therefore, the locally synchronized switch
is more scalable and more efficient than a global barrier, as confirmed
by our measurements.

Most of the features required by the synchronizing switch already
exist in basic wormhole routers. With a slight modification, the
routers of several distributed memory machines can support the
synchronizing switch. These hardware additions make the phased AAPC
practical for machines like Paragon or the SP-1 where the computation
and communication agents are not as tightly coupled as in the iWarp.

One could build a network architecture purely from synchronizing
switches, supporting all communication steps as subsets of AAPC.
However, our measurements of common communication steps show that
while dense communication steps would benefit from phased AAPC, many
sparse communication steps would lose a factor of two to three over a
conventional message passing architecture. A network architecture
optimized for both worlds can configure one set of virtual channels
(i.e. a pool) to implement the synchronizing switches for AAPC and
leave the remaining sets for message passing or other communication
methods. In this case, conventional message passing and phased AAPC
communication can co-exist, and the application or compiler can chose
the appropriate communication primitive.

Since programmers and compilers can frequently detect AAPC steps, AAPC
primitives can be utilized. The primitives execute significantly
faster on an architecture for optimized AAPC. The necessary support
should be considered in future network designs, and phased AAPC calls
should be included into standard message passing libraries.

%\nocite{MPI-STANDARD}

%\begin{itemize}
%\item Phased AAPC can be practical. Over 2 GB/s achieved.
%\item Synchronizing switch protoptype is more scalable and efficient
%that global synchronization
%\item With slight modification to basic wormhole router, one
%can implement the synchronization switch in hardware. This makes
%it practical for machines like Paragon where the computation and
%communication agents are not so tightly coupled. 
%\item Many common communication patterns are sparse and do not
%perform well as subsets as AAPC, so synchronizing switch alone is
%not a good idea.
%\item Many routers have multiple pools of virtual channels. Can adapt
%a single pool to implement the synchronizing switch. Then message
%passing and phased AAPC can co-exist.
%\end{itemize}
