Lecture notes from Scott Hudson,
Guest Lecture for 05-830 on January 25, 1999
Constraints
A general technique useful for describing and maintaining relationships
several important UI tasks
presentation
geometry management
automation of feedback aspects of presentation
(this gate should turn red when its fan-out limit is exceeded)
dialog management
maintaining highlight & enable status
application interface
and connections between application data and interface objects
multiple views
support for constructing interactive objects by aggregation
A constraint system consists of
a set of variables
- in our case these variables are typically embedded inside objects
things like:
size & position
choice of color or icon
visible/invisible
enabled/disabled
value of scrollbar
etc.
- in this treatment we will ignore this and assume that attrs just "float"
but always remember that these are values embedded in interactor tree
(or appl data structures) hence they affect display (and possibly how
input is handled)
a set of relationships that are to hold between the variables
the system is responsible for making sure constraints hold
under changes to the variables
under changes to the constraints
=> a declarative approach
say "what" is to happen
leave "how" to the system
if the system can do it, this is a lot less work on programmer
Unfortunately, general case of constraint solution is unsolvable
However restricted, but very useful restricted versions can be handled
User interfaces typically use one of two restricted classes
"one-way" constraints
A more general class of "multi-way" constraints
- don't have time to say much about these.
(For us...)
Relationships normally expressed by equations (expressing equality)
A = B + C
In the general case, if we change A, B, or C
system is responsible for finding changing values of one or both of
the other vars in order to maintain constraint
- note we normally have multiple constraints that must hold,
so job is a bit trickier than it might appear
In first restricted case, we only allow the information to flow in one
direction:
- one way constraints
A <- B + C
under changes to B or C, system will find a value of A that satisfies
constraints
Details of restrictions:
variables can only be the "target" of one constraint
- one defining constraint
variables which have a defining constraint may not be directly assigned
no cycles in dependency graph
- not everybody agrees on that part
Brad's systems make use of cycles
An algorithm to efficiently update one-way constraints
Some terminology:
will need to build a dependency graph
indicates which variables are dependent on which others
- note will often call variables "attributes"
Node in the graph for each attribute
Edge from A to B indicates that B is dependent on A
A = B + C yields two edges
B -> A
C -> A
Note this is data flow direction sometimes see this in other
("depends on" relation) direction
When we make a change to a value
(lets set aside changing the constraints for now)
This may affect other values "down stream" in the dependency graph
Use "grid-shaped" dependency graph for intuition
<>
if we change a value the set of values that might change fans out
in a wedge
Simple update algorithm:
when I change value, recompute everybody that depends on me
(if they change, they cause others to recompute, etc.)
=> data-driven algorithm
Problem:
/--->(B1)---\ /--->(B2)---\ ...
/ v / v /
(A1)------------>(A2)------------>(A3) ... (An)
\ ^ \ ^ \
\--->(C1)---/ \--->(C2)---/ ...
When we change A1:
eval B1, A2, and C1
B1 changes
eval A2
C1 changes
eval A2
Notice A2 gets several values before it settles down on final value
In fact its worse than several
A2 gets 3 values
A3 gets 9 (3 for each different value of A3)
An gets 3^n values!!
(that can take a while)
both breadth first and depth propagation are exponential
in this case breadth first is *only* 2^n, but others are worse
What do we do?
Need to respect topological order of the graph
Never compute the value of an attribute until *all* the attributes
it depends on have final values
Following works:
Topologically sort the graph
For each attribute A in topsort order ("lowest" to "highest")
evaluate A
topsort guarantees that things attribute depends on are all evaluated
before the attribute
"What you don't remember the topsort algorithm!?"
basically just a depth first traversal
But we are still doing too much work
<> (again)
only some of the attributes in the system could get new values
but above approach reevaluates everything
would like an incremental solution
just do the things inside the wedge
In fact, that may be more work than we need
incoming values of some attributes might change,
but same value results
example: a = b || c
b == true and c changes
a still has same value
<>
Would like to update only "influenced" attributes
the ones that change value, plus one more downstream
(if a parameter changes we can't tell in advance if value stays the
same or changes, so we have to evaluate one past point of actual
change)
not update all of "maybe_influenced"
But that is still more than we need
may not actually use all values
would like to be able to selectively enable/disable expensive features
consider: a = if do_expensive_op then expensive_op(x,y,z) else 0
If we go back to the wedge, we have an analogous situation
use of a value ("demanding" that value) may require a other attributes
again, fans out
but may not require all attributes in wedge
- don't need x,y,z or anything they depend on from above
<>
would like to just evaluate "needed" not "maybe_needed"
In fact what we want is intersection of "influenced" and "needed"
- actually a bit more complicated if you requested and modify
different attributes over time, but that gives the intuition
So, how do we do this?
Add a bit of bookkeeping
Put a boolean ("out_of_date"/"OOD") on each attribute
- indicates that value *might* be out of date wrt its equation
(will correspond essentially to the "maybe_influenced" set)
And a boolean ("pending") on each edge
- indicates that there is a changed value which has not been
propagated across that edge
(represents the "high water mark" of the "influenced" set)
For each change of an attribute (lets assume a static dep graph for now)
{
do a recursive walk and mark each reachable attribute as OOD
(if we reach an attribute that is already OOD stop)
set pending mark on all outgoing edges
}
For each requested value
if the value is OOD
{
set OOD false
recursively request values it depends on
if any incoming edges are marked pending
{
clear all incoming pending marks
evaluate the equation to get a new value
if the value has changed
mark each outgoing edge as pending
}
}
return the value
Actually, we only request attributes that the equation needs
example if-then-else never needs more than 2 of its values
=> intertwine evaluation and testing for pending
This algorithm is "partially optimal"
executes the minimal set of equations after any given change
may do slightly more total bookkeeping work than optimal in rare cases
- happens in mark OOD phase which is cheap
- lower bound result says it is optimal in all cases where
maybe_influence < 2^|influenced|
Multi-way constraints
Relaxing the restriction on information flow direction
A = B + C
can change any variables
- more powerful/expressive
- e.g. difficult to do some centering constraints in one-way
- e.g. regain symmetry.
system is responsible for finding other values
- this isn't always possible
(can be over constrained)
note: this can never happen in one-way's
- may be possible, but particular algorithm can't do it
most troubling:
- may be more than one way to do it
(under constrained)
arbitrary choices may be surprising
i.e., constrain a point to be at the midpoint of a line segment
can solve constraint after any change by making
all points coincident
=> solves constraint, but not what you want
- in general much less understandable behavior than one-way
(but you have to give up power for understandability)
Best solution to this so far: "constraint hierarchies"
- odd name, better thought of as "constraint strengths"
- don't have time to consider algorithms to update,
but they exist and are efficient within certain limitations
Opus
A more general visual notation for layout
Based on an underlying constraint-based system (Penguims)
Supports a nice general notation for layout based on constraints
Allows other aspects of appearance to be driven by constraints
- provides another way to do application interface
establish & automatically maintain relationship between `
appl data structures and displays
four basic constructs in notation:
frames
constraints
reference lines
interactor objects
Interactor objects are the only parts that appear in the final interface
- reasonable subset of the Artkit interactor library is supported
Each object is represented by a bounding box (+ "other properties")
each edge of bounding box represents one coordinate
- top, bottom, left, right
pretty much all geometry is covered this way
(only things like polygons and splines need more)
As before,
layout task comes down to size and position of these rectangles
each edge can either have a constraint that controls it or be "free"
visual convention: constrained edges are drawn double
- makes it easy to see where you can and can't add constraints
some objects have an intrinsic size => internal constraints
- in this version of system, bottom left is always constrained
- later we dynamically add second constraint once first established
non-geometric aspects of object
controlled by values or equations
entered in property sheet for the object
constraints are represented by arrows
<>
one-way constraints so directional
always goes from horiz to horiz or vert to vert
small box holds equation
$ in equation represents value at source
can put application objects/function into constraints
=> application interface
=> possibilities for semantic feedback
also a special un-anchored form of constraint with no source
Frames
<>
- don't appear in the final interface
used as a grouping unit
also the unit of composition (& reuse)
make hierarchy by "plugging" one frame into another
- composite frames vs. open frames
Frames support "reference lines"
analogy to reference lines used by draftsmen
very thin line drawn to align other things to, later erased
reference lines in Opus do the same
don't appear in final interface, but used to align things
reference lines come in several varieties
"free"
just floating, defined by a constraint like any other edge
min, max, average
do a computation over any number of incoming constraints
proportional
a fixed proportion of the frame they are within
- provides effect of Cardelli's attachment points
Examples of how to put this all together
Fig. 8 fixed size, stick to top left
Fig. 9 expand to fill all the available space
Fig. 10 stretch proportionally
Fig. 11 centered under
Notice that neither "outside-in" (top-down) or "inside-out" (bottom-up)
is sufficient for everything you want to do
e.g. Fig. 12 needs both
The path of least resistance is very important...
what does OPUS not do (well)?
doesn't handle number independent layout
- must know/specify all objects in advance
- can't talk about objects not created until run-time