	ECO: Efficient Collective Operations
	Beta release 0.1b
	Bruce Lowekamp and Adam Beguelin	
	School of Computer Science
	Carnegie Mellon University
	Pittsburgh, PA 15213

	(C) 1996 All Rights Reserved

NOTICE:

 Permission to use, copy, modify, and distribute this software and
 its documentation for any purpose and without fee is hereby granted
 provided that the above copyright notice appear in all copies and
 that both the copyright notice and this permission notice appear in
 supporting documentation.

 Neither Carnegie Mellon University nor the Authors make any
 representations about the suitability of this software for any
 purpose.  This software is provided ``as is'' without express or
 implied warranty.

 This research is sponsored in part by the Department of Defense
 Advanced Research Projects Agency and the National Science
 Foundation.


General notes:

All functions require "eco.h" to be included.

[] is used to represent vectors.  All functions are declared accepting
void * for vector arguments.

TYPE is one of
Eco_Double
Eco_Int
Eco_Float
Eco_Long
Eco_Char

For reduce and scan, OPERATION is one of
ECO_OP_ADD
ECO_OP_MULT
ECO_OP_MAX
ECO_OP_MIN
or a user-defined function registered with eco_register_function


All functions (except barrier and bcast) have multivector forms.  These are
identical to the base operations except the function call ends in a v (e.g.
eco_gather->eco_gatherv), the first argument is the (integer) number of
vectors, and the send and receive vector arguments are now 2d.

root is the index of the root process in the tid array which is returned
by eco_init (identical to the array that is passed unless a topology
is requested).  or, for some operations, it can be ECO_ALL, which is defined
in eco.h

-----------------------------------------------------------------------

eco_gather (IN send_vector[], IN int send_length, IN int send_stride,
	    IN int send_offset,
	    OUT recv_vector[], IN int recv_length, IN int recv_stride,
	    IN int root, IN TYPE)

send_vector: vector with data to be gathered
send_length: number of data elements in send_vector
send_stride: stride for data in send_vector
send_offset: position of send_vector in recv_vector
recv_vector: resulting vector (root)
recv_len: size of recv_vector (in terms of data elements) (root)
recv_stride: stride of data in recv_vector (root)
root: gathering process or ECO_ALL

gather takes all send vectors contributed by processes and combines them
to form recv_vector, which is only used at root, unless root=ECO_ALL, in 
which case all processes receive the recv_vector.  Overlapping send
ranges will produce unpredictable results.

send_offset should specify the destination address with recv_stride already
accounted for.  Note that length parameters should not include stride


-------------------------------------------------------------------------------
eco_scatter(IN send_vector[], IN int send_length, IN int send_stride,
	IN int lengths[], IN int offsets[],
	OUT recv_vector[], IN int recv_length, IN int recv_stride,
	IN int root, IN TYPE)

send_vector: data to be scattered (root)
send_length: number of data items (root)
send_stride: stride of data in send_vector (root)
lengths: number of items process i should receive (root)
offsets: offset of data process i should receive (root)
recv_vector: part of data for process 
recv_length: length of data for process 
recv_stride: stride of data in recv_vector
root: scattering process

scatter divides data input on root and distributes one chunk to each process
(including the root process).  It does check that output_vector is large
enough to store what is passed to it, although overlaps within send_vector
are not checked for.

Note that offsets should specify the source address with stride already
accounted for.  length parameters should not include stride

-------------------------------------------------------------------------------

eco_alltoall(IN send_vector[], IN int send_length, IN int send_stride,
	IN int send_lengths[], IN int send_offsets[],
	OUT recv_vector[], IN int recv_length, IN int recv_stride,
	IN int recv_lengths[], IN int recv_offsets[], TYPE)

send_vector: data to be scattered
send_length: number of data items in send_vector of send_vector
send_lengths: number of items process i should receive
send_offsets: offset of data process i should receive
recv_vector: data to be gathered
recv_length: number of data items to be gathered
recv_lengths: number of items to be received from process i
recv_offsets: offset in recv_vector at which to store data from process i

alltoall is essentially a combined scatter/gather where each process
sends distinct data to each other process.

Note that offsets should specify the source address with stride already
accounted for.  length parameters should not include stride


-------------------------------------------------------------------------------

eco_reduce(IN send_vector[], IN int length, IN int stride,
	   OUT recv_vector[], 
	   IN OP_TYPE operation, IN int root, TYPE)

send_vector: local contribution of data
length: number of data items in send_vector and recv_vector
stride: stride of send_vector and recv_vector
recv_vector: result (root)
operation: operation type
root: receiving process or ECO_ALL

reduce does operation across all copies of send_vector.  This operation is
elementwise, so recv_vector[i] = OPERATION across all send_vector[i].

the length parameter should not reflect stride

-------------------------------------------------------------------------------
eco_scan(IN send_vector[], IN int length, IN int stride
         OUT recv_vector[], IN OP_TYPE operation, TYPE)
   
send_vector: local contribution of data
length: number of data items in send_vector and recv_vector
stride: stride of send_vector and recv_vector
recv_vector: operation performed on first 0..i contributions
operation: operation to do on send_vector
   
scan implements an inclusive scan, such that the vector returned to 
process i is the result of applying operation to send_vector from
processes 0..i.  

-------------------------------------------------------------------------------

eco_barrier()

waits until all processes reach call before completing


-------------------------------------------------------------------------------

eco_bcast(IN int root)

Broadcasts the current send buffer from the root node to all nodes
(including root).  All processes must call this or other processes may not
receive the message.  Message is received into a new receive buffer,
as with pvm_recv.


-------------------------------------------------------------------------------

eco_init (IN/OUT int* p_i, IN int p, IN/OUT int tids[], char data_filename[],
	IN int topology)

p_i: pvm_mytid() = tids[p_i]  0<=p_i<p
p: number of tasks participating in collective communication
tids: array of tids.  returned in new order if topology is used
data_filename: network database filename (used by p_i=0)
	defaults to a compile time value if NULL
topology: topology to provide for local communication.  values:
		ECO_DEFAULT_TOPOLOGY
		ECO_RING_TOPOLOGY

eco_init is used to read in the database and generate patterns for
collective communication.  It must be called by all tasks.  If
topology is set to something other than ECO_DEFAULT_TOPOLOGY (0), tids
may be returned in a different order.


-------------------------------------------------------------------------------
eco_register_function(function)

registers function as a binary operation for reduce and scan.  returns
the (integer) identifier to be passed to reduce and scan

The function should be of type

void ECO_user_function(void* In, void* InOut, int length, int* info);

and should implement the operation

for(i=0;i<length;i++)
   InOut[i] = InOut[i] (+) In[i];

info is reserved for returning status, although it is currently not used.
(+) must be both associative and commutative

-------------------------------------------------------------------------------

eco_delete_function(int function_id)

deletes the function previously registered


