

                                                  about-series-optimization


                             Optimization

Series expressions are transformed into loops by pipelining them---the
computation is converted from a form where entire series are computed one
after the other to a form where the series are incrementally computed in
parallel.  In the resulting loop, each individual element is computed just
once, used, and then discarded before the next element is computed.  For
this pipelining to be possible, a number of restrictions have to be
satisfied.  Before looking at these restrictions, it is useful to consider
a related issue.

The composition of two series functions cannot be pipelined unless the
destination function consumes series elements in the same order that the
source function produces them.  Taken together, the series functions
guarantee that this will always be true, because they all follow the same
fixed processing order.  In particular, they are all `preorder'
functions---they process the elements of their series inputs and outputs in
ascending order starting with the first element.  Further, while it is easy
for users to define new series functions, it is impossible to define one
that is not preorder.

It turns out that most series operations can easily be implemented in a
preorder fashion, the most notable exceptions being reversal and sorting.
As a result, little is lost by outlawing non-preorder series functions.  If
some non-preorder operation has to be applied to a series, the series can
be collected into a list or vector and the operation applied to this new
data structure.  (This is inefficient, but no less efficient than what
would be required if non-preorder series functions were supported.)


                         Basic Restrictions

The transformation of series expressions into loops is required to occur at
some time before compiled code is actually run.  Optimization may or may
not be applied to interpreted code.  If any of the restrictions described
below are violated, optimization is not possible.  In this situation, a
warning message is issued at the time optimization is attempted and the
code is left unoptimized.  This is not a fatal error and does not prevent
the correct results from being computed.  However, given the large
improvements in efficiency to be gained, it is well worth fixing any
violations that occur.  This is usually easy to do.


*SUPPRESS-SERIES-WARNINGS*                                       [Variable]

If this variable is set (or bound) to anything other than its default value
of NIL, warnings about conditions that block the optimization of series
expressions are suppressed.


Before discussing the restrictions on series expressions, it is useful to
define precisely what is meant by the term SERIES EXPRESSION.  This term is
semantic rather than syntactic in nature.  Given a program, imagine it
converted from Lisp code into a data flow graph.  In a data flow graph,
functions are represented as boxes, and both control flow and data flow are
represented as arrows between the boxes.  Constructs such as LET and SETQ
are converted into patterns of data flow arcs.  Control constructs such as
IF and LOOP are converted into patterns of control flow arcs.  Suppose
further, that all loops have been converted into tail recursions so that
the graph is acyclic.

A series expression is a subgraph of the data flow graph for a program that
contains a group of interacting series functions.  More specifically, given
a call F on a series function, the series expression E containing it is
defined as follows.  E contains F.  Every function using a series created
by a function in E is in E.  Every function computing a series used by a
function in E is in E.  Finally, suppose that two functions G and H are in
E and that there is a data flow path consisting of series and/or non-series
data flow arcs from G to H.  Every function touched by this path (be it a
series function or not) is in E.

FOR OPTIMIZATION TO BE POSSIBLE, SERIES EXPRESSIONS HAVE TO BE STATICALLY
ANALYZABLE.  As with most other optimization processes, a series expression
cannot be transformed into a loop at compile time, unless it can be
determined at compile time exactly what computation is being performed.
This places a number of relatively minor limits on what can be written.
For example, for optimization to be possible the type arguments to
higher-order functions such as MAP-FN and COLLECTING-FN have to be quoted
constants.  Similarly, the numeric arguments to CHUNK have to be constants.
In addition, if FUNCALL is used to call a series function, the function
called has to be of the form (FUNCTION ...).

FOR OPTIMIZATION TO BE POSSIBLE, EVERY SERIES CREATED WITHIN A SERIES
EXPRESSION MUST BE USED SOLELY INSIDE THE EXPRESSION.  (If a series is
transmitted outside of the expression that creates it, it has to be
physically represented as a whole.  This is incompatible with the
transformations required to pipeline the creating expression.) To avoid
this problem, a series must not be returned as a result of a series
expressions as a whole, assigned to a free variable, assigned to a special
variable, or stored in a data structure.  A corollary of the last point is
that when defining new optimizable series functions, series cannot be
passed into &REST arguments.  Further, optimization is blocked if a series
is passed as an argument to an ordinary Lisp function.  Series can only be
passed to the predefined series functions and to new series functions
defined using the declaration OPTIMIZABLE-SERIES-FUNCTION.

FOR OPTIMIZATION TO BE POSSIBLE, SERIES EXPRESSIONS MUST CORRESPOND TO
STRAIGHT LINE COMPUTATIONS.  That is to say, the data flow graph
corresponding to a series expression cannot contain any conditional
branches.  (Complex control flow is incompatible with pipelining.)
Optimization is possible in the presence of standard straight-line forms
such as PROGN, FUNCALL, SETQ, LAMBDA, LET, LET*, and MULTIPLE-VALUE-BIND as
long as none of the variables bound are special.  There is also no problem
with macros as long as they expand into series functions and straight-line
forms.  However, optimization is blocked by forms that specify complex
control flow (i.e., conditionals IF, COND, etc., looping constructs LOOP,
DO, etc., or branching constructs TAGBODY, GO, CATCH, etc.).

In the first example below, optimization is blocked, because the IF form is
inside of the series expression.  However, in the second example,
optimization is possible, because although the IF feeds data to the series
expression, it is not inside the corresponding subgraph.  Both of the
expressions below produce the same value, however, the second one is much
more efficient.

(COLLECT (IF FLAG (SCAN X) (SCAN Y)))  ;warning message issued 
(COLLECT (SCAN (IF FLAG X Y))) 


                         Constraint Cycles

Even if a series expression satisfies all of the restrictions above, it
still may not be possible to transform the expression into a loop.  The
sole remaining problem is that if a series is used in two places, the two
uses may place incompatible constraints on the times at which series
elements should be produced.

The series expression below shows a situation where this problem arises.
The expression creates a series X of the elements in a list. It then
creates a normalized series by dividing each element of X by the sum of the
elements in X.  Finally, the expression returns the maximum of the
normalized elements.

(LET ((X (SCAN '(1 2 5 2))))           ; warning message issued 
  (COLLECT-MAX (#M/ X (SERIES (COLLECT-SUM X))))) => 1/2


The two uses of X in the expression place contradictory constraints on the
way pipelined evaluation must proceed.  COLLECT-SUM requires that all of
the elements of X be produced before the sum can be returned and SERIES
requires that its input be available before it can start to produce its
output.  However, #M/ requires that the first element of X be available at
the same time as the first element of the output of SERIES.  For pipelining
to work, this implies that the first element of the output of SERIES (and
therefore the output of COLLECT-SUM) must be available before the second
element of X is produced.  Unfortunately, this is impossible.

The essence of the inconsistency above is the cycle of constraints used in
the argument.  This in turn stems from a cycle in the data flow graph
underlying the expression (see Figure 1).  In Figure 1, function calls are
represented by boxes and data flow is represented by arrows.  Simple arrows
indicate the flow of series values and cross hatched arrows indicate the
flow of non-series values.

  |------|   /------------------------------->|-------|      |------|
 -| scan |---     |------|      |------|      |  #M/  |----->| max  |-
  |------|   \--->| sum  |+++++>|series|----->|-------|      |------|
                  |------|      |------|  

        Figure 1: A Constraint Cycle in a Series Expression

Given a data flow graph corresponding to a series expression, a CONSTRAINT
CYCLE is a closed loop of data flow arcs that can be traversed in such a
way that each arc is traversed exactly once and no non-series arc is
traversed backwards.  (Series data flow arcs can be traversed in either
direction.)  A constraint cycle is said to PASS THROUGH an input or output
port when exactly one of the arcs in the cycle touches the port.  In Figure
1, the data flow arcs touching SCAN, SUM, SERIES, and #M/ form a constraint
cycle.  Note that if the output of SCAN were not a series, this loop would
not be a constraint cycle, because there would be no valid way to traverse
it.  Also note that while the constraint cycle passes through all the other
ports it touches, it does not pass through the output of SCAN.

Whenever a constraint cycle passes through a non-series output, an argument
analogous to the one above can be constructed and therefore pipelining is
impossible.  When this situation arises, a warning message is issued
identifying the problematical port and the cycle passing through it.  For
instance, the warning triggered by the example above states that the
constraint cycle associated with SCAN, COLLECT-SUM, SERIES, and #M/ passes
through the non-series output of COLLECT-SUM.

Given this kind of detailed information, it is easy to alleviate the
problem.  To start with, every cycle must contain at least one function
that has two series data flows leaving it.  At worst, the cycle can be
broken by duplicating this function (and any functions computing series
used by it).  For instance, the example above can be rewritten as shown
below.

(LET ((X (SCAN '(1 2 5 2))) 
      (SUM (COLLECT-SUM (SCAN '(1 2 5 2))))) 
  (COLLECT-MAX (#M/ X (SERIES SUM)))) 
 => 1/2


It would be easy enough to automatically apply code copying to break
problematical constraint cycles.  However, this is not done for two
reasons.  First, there is considerable virtue in maintaining the property
that each function in a series expression turns into one piece of
computation in the loop produced.  Users can be confident that series
expressions that look simple and efficient actually are simple and
efficient.  Second, with a little creativity, constraint problems can often
be resolved in ways that are much more efficient than copying code.  In the
example above, the conflict can be eliminated efficiently by interchanging
the operation of computing the maximum with the operation of normalizing an
element.

(LET ((X (SCAN '(1 2 5 2)))) 
  (/ (COLLECT-MAX X) (COLLECT-SUM X))) => 1/2


The restriction that optimizable series expressions cannot contain
constraint cycles that pass through non-series outputs places limitations
on the qualitative character of optimizable series expressions.  In
particular, they all must have the general form of creating some number of
series using scanners, computing various intermediate series using
transducers, and then computing one or more summary results using
collectors.  The output of a collector cannot be used in the intermediate
computation unless it is the output of a separate subexpression.

It is worthy of note that the last expression above fixes the constraint
conflict by moving the non-series output out of the cycle, rather than by
breaking the cycle.  This illustrates the fact that constraint cycles that
do not pass through non-series outputs do not necessarily cause problems.
They cause problems only if they pass through OFF-LINE ports.

A series input port or series output port of a series function is ON-LINE
if and only if it is processed in lock step with all the other on-line
ports as follows:  The initial element of each on-line input is read, then
the initial element of each on-line output is written, then the second
element of each on-line input is read, then the second element of each
on-line output is written, and so on.  Ports that are not on-line are
off-line.  If all of the series ports of a function are on-line, the
function is said to be on-line; otherwise, it is off-line.  (The above
extends the standard definition of the term `on-line' so that it applies to
individual ports as well as whole functions.)

If all of the ports a cycle passes through are on-line, the lock step
processing of these ports guarantees that there cannot be any conflicts
between the constraints associated with the cycle.  However, passing
through an off-line port leads to the same kinds of problems as passing
through a non-series output.

Most of the series functions are on-line.  In particular, scanners and
collectors are all on-line as are many transducers.  However, the
transducers in Section on complex transducers are off-line.  In particular,
the series inputs of CATENATE, CHOOSE-IF, CHUNK, EXPAND, MASK, MINGLE,
POSITIONS, and SUBSERIES along with the series outputs of CHOOSE, SPLIT,
and SPLIT-IF are off-line.

In summary, the fourth and final restriction is that FOR OPTIMIZATION TO BE
POSSIBLE, A SERIES EXPRESSION CANNOT CONTAIN A CONSTRAINT CYCLE THAT PASSES
THROUGH A NON-SERIES OUTPUT OR AN OFF-LINE PORT.  Whenever this restriction
is violated a warning message is issued.  Violations can be fixed either by
breaking the cycle or restructuring the computation so that the offending
port is removed from the cycle.

                    Defining New Series Functions

New functions operating on series can be defined just as easily as new
functions operating on any other data type.  However, expressions
containing these new functions cannot be transformed into loops unless a
complete analysis of the functions is available.  Among other things, this
implies that the definition of a new series function must appear before its
first use.


                             Declarations

A key feature of Lisp is that variable declarations are strictly optional.
Nevertheless, it is often the case that they are necessary in situations
where efficiency matters.  Therefore, it is important that it be POSSIBLE
for programmers to provide declarations for every variable in a program.
The transformation of series expressions into loops presents certain
problems in this regard, because the loops created contain variables not
evident in the original code.  However, if the information described below
is supplied by the user, appropriate declarations can be generated for all
of the loop variables created.

All the explicit variables that are bound in a series expression (for
example, by a LET that is part of the expression) should be given
informative declarations making use of the type specifier (SERIES
ELEMENT-TYPE) where appropriate.

Informative types should be supplied to series functions (such as SCAN and
MAP-FN) that have type arguments.  When using SCAN it is important to
specify the type of element in the sequence as well as the sequence itself
(for example, by using (VECTOR * INTEGER) as opposed to merely VECTOR).
The form (LIST ELEMENT-TYPE) can be used to specify the type of elements in
a list.

If it is appropriate to have a type more specific than (SERIES T)
associated with the output of #M, #Z, SCAN-ALIST, SCAN-FILE, SCAN-HASH,
SCAN-LISTS-OF-LISTS-FRINGE, SCAN-LISTS-OF-LISTS, SCAN-PLIST, SERIES,
LATCH, or CATENATE, then the form THE, must be used to specify this type.

Finally, if the expression computing a non-series argument to a series
variable is neither a variable nor a constant, THE must be used to specify
the type of its result.

For example, the declarations in the series expressions below are
sufficient to ensure that every loop variable will have an accurate
declaration.

(COLLECT-LAST (CHOOSE-IF #'PLUSP (SCAN '(LIST INTEGER) DATA))) 

(COLLECT '(VECTOR * FLOAT) 
         (MAP-FN 'FLOAT #'/ 
                 (SERIES (THE INTEGER (CAR DATA))) 
                 (THE (SERIES INTEGER) (SCAN-FILE F))))


The amount of information the user has to provide is reduced by the fact
that this information can be propagated from place to place.  For instance,
the variable holding the output of CHOOSE-IF holds a subset of the elements
held by the input variable.  As a result, it is appropriate for it to have
the same type.  When defining a new series function, the type specifier
SERIES-ELEMENT-TYPE can be used to indicate where type propagation should
occur.



     SEE ALSO
     about-series
     about-generators
     optimizable-series-function
     off-line-port
     series-element-type

;Copyright 1989 by the Massachusetts Institute of Technology,
;Cambridge, Massachusetts.

;Permission to use, copy, modify, and distribute this software and its
;documentation for any purpose and without fee is hereby granted,
;provided that this copyright and permission notice appear in all
;copies and supporting documentation, and that the name of M.I.T. not
;be used in advertising or publicity pertaining to distribution of the
;software without specific, written prior permission. M.I.T. makes no
;representations about the suitability of this software for any
;purpose.  It is provided "as is" without express or implied warranty.

;    M.I.T. DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
;    ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL
;    M.I.T. BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR
;    ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
;    WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
;    ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
;    SOFTWARE.



