\section{Concluding remarks}

We developed techniques for flexible barrier synchronization on
private--memory parallel machines, where arbitrary disjoint subsets of
processors can synchronize independently and in parallel, and where
only the name of each barrier and the number of participating
processors are known at compile time.  We introduced two general
communication models: the {\em bounded buffer broadcast model}, based
on a global broadcast network, and the {\em anonymous destination
message passing model}, based on a shared general--purpose
communication network. We developed generic algorithms based on these
models.  We adapted these generic algorithms to a ring of iWarp
processors analyzed the time and message complexities of the resulting
algorithms, and compared the measured performance on real iWarp
hardware with the performance predicted by the complexity models.

There are number of results from this work. First is the $O(\sqrt{N})$
running time of the ring--based barrier, rather than the $O(N)$
running time we would initially expect. Second is the accuracy of the
performance models, which accurately predict the running times of the
algorithms on real iWarp hardware. Third is the low price we pay for
the flexibility of the barrier algorithms. Even though arbitrary
subsets of processors are allowed to synchronize, with only the name
and number of processors involved in each barrier known at compile
time, processors that do participate in barriers incur only a small
overhead, and processors that do not participate incur no overhead.

Efficient barrier synchronization is typically associated with
dedicated global hardware resources, such as a broacast bus. However,
in this investigation, the ring-based barriers, which used a
general--purpose communication system, consistently outperformed the
broadcast--based barriers, which used a global broadcast network. This
leads us to the conclusion that efficient and flexible barrier
synchronization can be supported by a general--purpose communication
system, provided sufficient care is taken to ensure fast message
transfers between processors and fast direct access to the
communication system on each processor.



