05-830, User Interface
Software, Spring, 1999
Lecture 14, April 12,
Copyright © 1999 - Brad A. Myers
Lecture . . .
- Direct Manipulation allows properties to be set by directly
moving objects with the mouse and setting properties
- Example: interface builders, Visual Basic
- Limited to static parts of the interface
- No way to point at objects that will be drawn by the
- How set the color of an object in Visual Basic at run time?
- Demonstrational Tools allow the user to operate on
example objects which represent objects that are created at run-time.
- For example, the system must find out that the size of the boxes
depends on the actual labels typed by the user and where the lines attach.
- In general, demonstrational systems allow the user to operate on examples,
and then generalizes to produce a general-purpose procedure or prototype.
- draw an example of the objects that will be created at
- draw objects in approximately the right places, and systems creates general
- This is a hard problem, which is why you don't see many commercial
products that do this.
- There are also demonstrational systems for other domains (not
for creating UIs):
- Our group:
- Text editing (Tourmaline, Andy Werth INI MS thesis)
- Creating shell programs (Francesmary Modugno's PhD thesis on "Pursuit")
- Creating custom business charts and graphs (patent on this technology)
- WWW cgi and "agent" scripts (promising area for future research)
- Text editing (renumber by example, styles)
- Graphical editing (graphical procedures)
- Determining loops in HyperCard ("Eager")
- Reference: Allen Cypher, Watch What I Do, MIT Press
- Demonstrational techniques expand how much of the interface
can be specified interactively.
- And Interactive editors are much faster to use than programming
- Frameworks improve productivity by factors of 3 to 5, interactive
tools by factors of 10 to 50!
- It might take an hour to draw an interface interactively, compared to
days to program it.
- Because they are faster, this promotes rapid prototyping
- It is much more natural to specify the graphical parts of applications
using a graphical editor.
- Because they do not require programming skills, graphic designers can
design the graphical parts of the interface.
Examples (of uses to create UIs)
- (chronological order)
- Peridot (1986-88)
- Myers B. "Creating User Interfaces Using
Programming-by-Example, Visual Programming, and Constraints," ACM Transactions
on Programming Languages and Systems. vol. 12, no. 2, April, 1990. pp.
- Myers B., Creating User Interfaces by Demonstration, Academic
Press, San Diego, 1988.
- Myers B., "Creating Interaction Techniques by Demonstration," IEEE
Computer Graphics and Applications, Vol. 7, No. 9, IEEE, September 1987,
pp. 51 - 60.
- First demonstrational tool, and it used by-example techniques to allow
the creation of new widgets.
- - From the drawings, it infers
- Graphical constraints among the objects, such as that the boxes
should be the same size as the text.
- control structures such as iteration over all the items in a menu
- how the mouse affects the graphics, such as that the check mark should
follow the mouse.
- feedback: question and answer
- video (8 min)
- Lapidary (1989-1993)
- Myers B., Vander Zanden B. and Dannenberg R., "Creating Graphical
Interactive Application Objects by Demonstration," Proceedings of the
ACM Symposium on User Interface Software and Technology, UIST'89,
Williamsburg, November 1989, pp. 95 - 104.
- Brad Vander Zanden and Brad A. Myers. "Demonstrational and Constraint-Based
Techniques for Pictorially Specifying Application Objects and Behaviors,"
ACM Transactions on Computer-Human Interaction. vol. 2, no. 4, Dec,
1995. pp. 308-356.
- Extends Peridot to allow the creation of application-specific graphical
objects, like nodes in a graphics editor.
- Uses less inferencing and more dialog boxes
- Is "real" and you get it as part of the Garnet distribution
- can only demonstrate "syntactic" parts of application
- hard to set up correct constraints
- video (3 min)
- Druid (1990)
- Singh G., Kok C. and Ngan T., "Druid: A System for Demonstrational
Rapid User Interface Development," Proceedings of !the Symposium on User
Interface Software and Technology, UIST'90, Snowbird, October 1990, pp.
167 - 177.
- Use Peridot-style inferencing to place widgets in an Interface Builder
- Infers graphical constraints
- Feedback: dialog box question and answer
- Also by-demonstration for initial values of widgets, and some actions
like popping up other windows
- DEMO and DEMO II (1991, 1992)
- David A. Wolber and Gene L. Fisher, "Demonstrational Technique
for Developing Interfaces with Dynamically Created Objects," Proceedings
UIST'91: ACM SIGGRAPH Symposium on User Interface Software and Technology,
Nov, 1992, Monterey, CA, pp. 89-97.
- Gene L. Fisher, Dale E. Busse, and David A. Wolber, "Adding Rule-Based
Reasoning to a Demonstrational Interface Builder," Proceedings UIST'92:
ACM SIGGRAPH Symposium on User Interface Software and Technology, Nov,
1991, pp. 221-230.
- DEMO was first system to support dynamic creation of objects.
- Infers graphical relationships by examples of edits
- DEMO-II adds extensive inferencing of graphical constraints from examples
- Marquise (1993-1994)
- Myers B., McDaniel, R. and Kosbie, D.. "Marquise: Creating Complete
User Interfaces by Demonstration," Proceedings CHI'94: Human Factors in
Computing Systems. Amsterdam, The Netherlands, April 24-29, 1993. pp.
- Go back to doing more by demonstration, and just show the way that the
interface should operate.
- In particular, demonstrate when the behaviors should start and
what the feedback looks like.
- mouse button does one of 10 things, depending on where press
and global mode.
- Demonstrate both behavior and conditions
- Built-in support for palettes and modes.
- video (3 min)
- InferenceBear & Grizzly Bear (1994-1996)
- Martin Frank and Jim Foley, "Model-Based User Interface Design
by Example and By Interview," Proceedings UIST'93: ACM Symposium on User
Interface Software and Technology. Atlanta, GA, Nov. 3-5, 1993, pp.
- Martin Frank, Model-Based User Interface Design by Demonstration and
By Interview. PhD Thesis, Georgia Tech, 1996.
- (Discussed his "Elements, Events & Transitions (EET) language in
the event-language lecture)
- User control through dialog boxes, edit using textual language: EET
- Snapshots of before and after
- Multiple examples
- More positive examples to cause generalization
- Negative examples to specify exceptions
- Pavlov (1995-present)
- David Wolber, "Pavlov: Programming by Stimulus-Response
Demonstration," Proceedings CHI'96, Human Factors in Computing Systems.
April 1996. pp. 252-259
- Stimulus from mouse or time-based
- Score editor for feedback and editing
- video (2 min)
- Gamut (1996 - )
- PhD thesis of Rich McDaniel.
- Richard G. McDaniel and Brad A. Myers. "Building Applications Using Only
Demonstration," IUI'98: 1998 International Conference On Intelligent User
Interfaces, January 6-9, 1998, San Francisco, CA. pp. 109-116.
- Richard G. McDaniel and Brad A. Myers,
"Getting More Out Of Programming-By-Demonstration."
Proceedings CHI'99: Human Factors in
Computing Systems. Pittsburgh, PA, May 15-20, 1999. To appear.
- Domain: "board games" and educational software
- Goal: new interaction techniques so can infer more complex behaviors
- E.g., how a piece can move in Monopoly / Chess
- Reduce number of modes
- New interaction techniques to provide hints
- "Do Something!", "Stop That", Hint highlighting, Temporal Ghosts,
Guide objects, Deck
of Playing Cards, etc.
- Better inferencing algorithms
- video (8 min)
- How intelligent is enough?
- AI problem
- Techniques for feedback and editing
- Combining inferencing with direct editing of the code
- A "really" successful product using this technology
Back to 05-830 main page