Schedule and Readings

Num Date Class Content
1 Tuesday,

Introduction to Course; What are "Software Structures for User Interfaces"; and Why are user interfaces hard to design and implement?

Slides for Lecture 1
Video for Lecture 1 on Canvas

Required Readings:

  • Brad A. Myers. "Challenges of HCI Design and Implementation," ACM Interactions. vol. 1, no. 1. January, 1994. pp. 73-83. ACM DL
    Discusses why user interfaces are important, and why they are hard to design and implement.

Recommended Reading:

  • Brad A. Myers. "A Brief History of Human Computer Interaction Technology." ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54. ACM DL

Start on Homework 1 - html, CSS, and JavaScript

2 Thursday,

Review of HTML and CSS

Slides for Lecture 2
Video for Lecture 2 on Canvas

Required Readings:

3 Tuesday,

Review of JavaScript

Slides for Lecture 3
Video for Lecture 3 on Canvas

4 Thursday,

Input 1: Conventional Input Models for Handling Input Events

Slides for Lecture 4
Video for Lecture 4 on Canvas

Required Readings:

  • The JavaScript resources listed above will cover event handling for Input Events, DOM Events, etc., but here are some extra resources:

Course add Deadline is 9/14/2020.

Turn in Homework 1.
Start on Homework 2 - Input Handling

5 Tuesday,

Input 2: Declarative input models; "Interactor" (Behavior) Objects in Garnet and Amulet

Slides for Lecture 5
Video for Lecture 5 on Canvas

Required Readings:

  • Brad A. Myers. "A New Model for Handling Input," ACM Transactions on Information Systems, vol. 8, No. 3. July, 1990. pp. 289-320. ACM DL Reference.
6 Thursday,

Output 1: Basic 2D Computer Graphics

Slides for Lecture 6
Video for Lecture 6 on Canvas

Required Readings:

7 Tuesday,

Output 2: Graphics Models: Structured Graphics; Multiple Hierarchies; Object-Oriented Techniques

Slides for Lecture 7
Video for Lecture 7 on Canvas

Required Readings:

  • Brad A. Myers, Richard G. McDaniel, Robert C. Miller, Alan Ferrency,  Andrew Faulring, Bruce D. Kyle, Andrew Mickish, Alex Klimovitski, and Patrick Doane. "The Amulet Environment: New Models for Effective User Interface Software Development", IEEE Transactions on Software Engineering, Vol. 23, no. 6. June, 1997. pp. 347-365. local pdf or IEEE DL.

Optional Readings:

  • David S. Kosbie, Brad Vander Zanden, Brad A. Myers, Dario Giuse. "Automatic Graphical Output Management", in Brad A. Myers, editor. The Garnet Compendium: Collected Papers, 1989-1990. Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-90-154, August, 1990. pp. 30-43. pdf.

Turn in Homework 2.
Start on Homework 3 - Graphics using SVG and Canvas

8 Thursday,

UI Implementation Concepts: Callbacks, Resources, Widget Hierarchies, Geometry Management

Slides for Lecture 8
Special Slides: SSUI-hw2-3hints (on Canvas)
Video for Lecture 8 on Canvas (including the discussion of the special slides)

Optional Readings:

9 Tuesday,

UI Software Models: Lexical-Syntax-Semantics, Seeheim Model, MVC, Design Patterns, State Diagrams

Slides for Lecture 9
Video for Lecture 9 on Canvas

Required Readings:

  • Glenn Krasner and Stephen T. Pope, "A Cookbook for Using the Model-View-Controller User Interface Paradigm in Smalltalk-80", Journal of Object-Oriented Programming (JOOP). August-September, 1988. vol. 1, no. 3. pp. 26-49. pdf scan at UCI

Optional Readings:

  • Mark Green, "A Survey of Three Dialogue Models," ACM Transactions on Graphics (5)3, Jul, 1986, pp 244-275. ACM DL Reference
  • William Buxton, "Lexical and Pragmatic Considerations of Input Structures," Computer Graphics, January, 1983, (17)1, pp. 31-37. (lexical, syntactic, semantic, etc.). html
  • chapter 10 from Olsen's book: "Editing Dialog Models". User Interface Management Systems: Models and Algorithms.Morgan Kaufmann; (November 1991) (pdf of the chapter (CMU only), also avaialble in Special Course Readings on Canvas), or Amazon listing for book.
10 Thursday,

What is "User Interface Software", Overview of UI Software and Tools

Slides for Lecture 10
Video for Lecture 10 on Canvas

Optional Readings:

  • Brad A. Myers. "Graphical User Interface Programming," chapter 48 of Computer Science Handbook -- Second Edition. Allen B. Tucker, editor in chief. Boca Raton, FL: Chapman & Hall/CRC Press, Inc., 2004. pp. 48-1 - 48-29. pdf  (This is a revision of Brad A. Myers. "User Interface Software Tools," ACM Transactions on Computer-Human Interaction. vol. 2, no. 1, March, 1995. pp. 64-103. ACM DL Reference)
  • Brad A. Myers, Scott Hudson and Randy Pausch. "Past Present and Future of User Interface Software Tools," ACM Transactions on Computer Human Interaction, (Mar. 2000). Volume 7, Issue 1. pp. 3-28. ACM DL Reference.
11 Tuesday,

React for JavaScript

Slides for Lecture 11
Video for Lecture 11 on Canvas

Required Readings:

Optional Readings:

Turn in Homework 3.
Start on Homework 4 - Using React

12 Thursday,

Some Other Web Toolkits - jQuery, AngularJS, Vue, Bootstrap, etc.

Slides for Lecture 12
Video for Lecture 12 on Canvas

Required Readings:

Course Drop Deadline is 10/12/2020.

13 Tuesday,

Evaluation of APIs and UI Tools, API Usability, Cognitive Dimensions

Slides for Lecture 13
Video for Lecture 13 on Canvas

Required Readings:

  • Brad A. Myers, Andrew J. Ko, Thomas D. LaToza, and YoungSeok Yoon. "Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools," IEEE Computer, Special issue on UI Design, 49, issue 7, July, 2016, pp. 44-52. IEEE DL or local pdf.
  • Brad A. Myers and Jeffrey Stylos, "Improving API Usability", Communications of the ACM, vol 59, No. 6, June, 2016, pp. 62-69, ACM DLhtml or local pdf.
  • Jakob Nielsen, Heuristic Evaluation. On line in html format. Includes list of 10 heuristics.
  • Green, T.R.G. and Petre, M., “Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework.Journal of Visual Languages and Computing, 1996. 7(2): pp. 131-174. ScienceDirect.
    Includes an analysis of HyperCard

Recommended Readings:

  • Jeffrey Stylos and Brad Myers, "Mapping the Space of API Design Decisions," 2007 IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC'07. Sept 23-27, 2007, Coeur d'Alene, Idaho. pp. 50-57. ieee or local pdf
  • Brian Ellis, Jeffrey Stylos, and Brad Myers. "The Factory Pattern in API Design: A Usability Evaluation". International Conference on Software Engineering (ICSE'2007). May 20-26, 2007. Minneapolis, MN. pp. 302-312. ACM DL or local pdf
  • T.R.G. Green, Cognitive Dimensions of Notations. Cognitive Dimensions of Notations Resource Site
14 Thursday,

Command Objects & Support for Undo

Slides for Lecture 14
Video for Lecture 14 on Canvas

Required Readings:

  • Brad A. Myers and David Kosbie. "Reusable Hierarchical Command Objects," Proceedings CHI'96: Human Factors in Computing Systems. Vancouver, BC, Canada. April 14-18, 1996. ACM DL Reference
    (note: the "hierarchical" part is not necessary to know)

Optional Readings:

  • Command Objects in Amulet (note: only the overall design is relevant, not the details)
  • Brad A. Myers. "Scripting Graphical Applications by Demonstration," Proceedings CHI'98: Human Factors in Computing Systems. Los Angeles, CA, April 18-23, 1998. pp. 534-541. ACM DL Reference and local pdf and video. (Topaz)
15 Tuesday,

Toolkit support for Gestural Input Techniques, Handwriting Recognition

Slides for Lecture 15
Video for Lecture 15 on Canvas

Required Readings:

  • J.O. Wobbrock, AD Wilson, Y Li. "Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes", In UIST '07, p. 159-168. ACM DL Ref

Optional Readings:

Take-home Midterm Exam

16 Thursday,

Constraints and Data Bindings, 1

Slides for Lecture 16
Video for Lecture 16 on Canvas

Required Readings:

  • Scott E. Hudson. "Incremental attribute evaluation: a flexible algorithm for lazy update," ACM Transactions on Programming Languages and Systems, Volume 13 , Issue 3 (1991), Pages 315-341.  ACM DL Reference

Optional Readings:

  • Brad Vander Zanden, Brad A. Myers, Dario Giuse and Pedro Szekely. "Integrating Pointer Variables into One-Way Constraint Models," ACM Transactions on Computer-Human Interaction. vol. 1, no. 2, June, 1994. pp. 161-213. ACM DL Reference
  • Brad Vander Zanden, "An Incremental Algorithm for Satisfying Hierarchies of Multi-way, Dataflow Constraints", ACM Transactions on Programming Languages and Systems, 18(1), January, 1996. pp. 30-72. ACM DL Reference

Turn in Homework 4. (postponed deadline)
Start on Homework 5 - Implement Undo using Command Objects

Mid-Semester Grades Due on 10/26 by 4 pm

17 Tuesday,

Constraints 2: Implementations

Slides for Lecture 17
Video for Lecture 17 on Canvas

Please fill out the mid course survey to help improve the course: Thanks!

18 Thursday,

Constraints 2: Implementations, cont.

Slides for Lecture 18 (same slides as for lecture 17)
Video for Lecture 18 on Canvas

Required Readings:

  • (Same as lecture 16)

Optional Readings:

  • Bjorn N. Freeman-Benson, John Maloney and Alan Borning. "An incremental constraint solver," Communications of the ACM . Volume 33 , Issue 1 (1990). Pages 54-63. ACM DL Reference
19 Tuesday,

Connecting to the backend - Web Services and Databases

Slides for Lecture 19
Video for Lecture 19 on Canvas

Required Readings:

US Election Day

Turn in Homework 5 (note: due on 11/4).
Start on Homework 6 - Connecting to a Backend

20 Thursday,

Interactive Tools: Prototypers (HyperCard, Director, Visual Basic, Balsamiq), Interface Builders, Sketching Tools

Slides for Lecture 20
Video for Lecture 20 on Canvas

Required Readings:

  • James Landay and Brad A. Myers. "Interactive Sketching for the Early Stages of User Interface Design," Proceedings CHI'95: Human Factors in Computing Systems. Denver, CO. May, 1995. pp. 43-50. (html) or ACM DL Ref

Optional Readings:

  • Brad A. Myers. "Separating Application Code from Toolkits:  Eliminating the Spaghetti of Call-Backs,"  ACM Symposium on User Interface Software and Technology: UIST'91, Hilton Head, SC, Nov. 11-13, 1991. pp. 211-220. ACM DL Reference
  • Walker, M., Takayama, L., and Landay, J. (2002) High-fidelity or low-fidelity, paper or computer medium? Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (in press). 
  • James A. Landay and Brad A. Myers. Just Draw It! Programming by Sketching Storyboards.   Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-95-199 and Human Computer Interaction Institute Technical Report CMU-HCII-95-106. November, 1995. html or pdf

Semester Course Withdrawal Grade Deadline is 11/9/20

Start planning Final Project Topics

21 Tuesday,

Software Engineering for UIs: How to create high-quality UIs in the context of Agile and other software development processes

Slides for Lecture 21
Video for Lecture 21 on Canvas

Required Readings:

  • Budwig, M., Jeong, S. and Kelkar, K. When user experience met agile: a case study. In Proceedings of CHI'2009: The 27th international conference extended abstracts on Human factors in computing systems (Boston, MA, USA, 2009). ACM, 3075-3084. ACM DL PDF
22 Thursday,

Simple User Interface Toolkits and End-User Programming for UIs; Low-Code / NoCode

  • Examples: Scratch, Alice, AirTable,, Zapier, LabView, IFTTT, (older: Yahoo Pipes)

Slides for Lecture 22
Video for Lecture 22 on Canvas

Required Readings:

  • Conway, M., Audia, S., Burnette, T., Cosgrove, D., Christiansen, K., Deline, R., et al. (2000, Apr 1-6). Alice: Lessons Learned from Building a 3D System For Novices. Paper presented at the Proceedings of CHI 2000, The Hague, The Netherlands. ACM DL Ref, or citeseer ref

Optional Readings:

  • Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344. ACM DL Ref, or citeseer ref
  • J.F. Pane, B.A. Myers, and L.B. Miller, "Using HCI Techniques to Design a More Usable Programming System," 2002 IEEE Symposia on Human Centric Computing Languages and Environments (HCC'2002). Arlington, VA, September 3-6, 2002. pp. 198-206. local pdf. (Winner Most Influential Paper Award for important influences on VL/HCC research or commerce over the last 10+/-1 years in 2012.)
  • Brad A. Myers, John F. Pane and Andy Ko, "Natural Programming Languages and Environments". Communications of the ACM. 47(9), pp. 47-52. ACM DL Ref

Turn in Homework 6 (extended until 11/16/2020)
Start on Final Project

23 Tuesday,

Toolkits for building speech/conversational/chatbot User Interfaces, and Visualizations

  • Speech Examples: DialogFlow, Microsoft Bot Network, Voiceflow, ...
  • Viz Examples: Prefuse, Flare, Improvise, VTK, Excel, Many Eyes, Google Charts, Tableau, VizQL, ggplot2, Protovis, D3

Slides for Lecture 23
Video for Lecture 23 on Canvas

Required Readings:

Optional Readings:

24 Thursday,

Toolkit support for Assistive and Accessible Interfaces; Web Accessibility

Slides for Lecture 24
Video for Lecture 24 on Canvas

Required Readings:

  • W3C, "Introduction to Web Accessibility," on-line
  • Zimmermann, G., Vanderheiden, G. Accessible design and testing in the application development process: considerations for an integrated approach. Univ Access Inf Soc 7, 117–128 (2008).

Optional Readings:

Please fill out the survey about homeworks 5 and 6 to help improve the course: Thanks!

25 Tuesday,

Toolkit support for Ubiquitous Computing, Context Awareness and Multiple users

Slides for Lecture 25
Video for Lecture 25 on Canvas

Required Readings:

X Thursday,


All classes afterwards online only

26 Tuesday,
Jason Hong's picture

Guest lecture: Jason Hong:
UI Programming Considering Privacy and Security: interfaces that respect user privacy; data flows that are problematic; "deceptive patterns" in user interaction

Slides for Lecture 26
Video for Lecture 26 on Canvas

Required Readings:

27 Thursday,

Animation in Toolkits

Slides for Lecture 27
Video for Lecture 27 on Canvas

Required Readings:

  • Bay-Wei Chang and David Ungar. “Animation: From Cartoons to the User Interface,” UIST'93: ACM SIGGRAPH Symposium on User Interface Software and Technology, Atlanta, GA, Nov, 1993. pp. 45-55. ACM DL Reference

Optional Readings:

  • Brad A. Myers, Robert C. Miller, Rich McDaniel, and Alan Ferrency, "Easily Adding Animations to Interfaces Using Constraints." ACM Symposium on User Interface Software and Technology, UIST'96, November 6-8, 1996. Seattle, WA. pp. 119-128. ACM DL Reference
28 Tuesday,

Toolkits for 3D Programming and the UIs of Games

Slides for Lecture 28
Video for Lecture 28 on Canvas

Optional Readings:

  • Richard Fine, "Going deep with IMGUI and Editor Customization", December 22, 2015, blog post -- discusses the event loop for Unity's old immediate-mode GUI system
  • J. Blow, “Game development: Harder Than You Think”, ACM Queue, ACM, New York, 2004, pp. 29-37. ACM DL REF
  • Take a look at standard game engines of today: Unity Game EngineUnreal Engine, and (for non-programmers) GameMaker. There are dozens of other game engines available.
  • OpenGL is the graphics library standard, and it is open source. However, it is highly complex. For a gentler introduction and some videos demonstrating its capability, look at a javascript-based wrapper for OpenGL called three.js.

Please fill out the class survey:, and the official CMU survey.

29 Thursday,

Presentation of Final Projects

My intro slides for Lecture 29

Six presentations, one from each group. Tentative order:

  1. Viz
  2. To-Do List Chatbot
  3. Remote HQ
  4. State Diagram
  5. DataSpeak
  6. UI Craft

Please use this form to review each group's presentation (except your own group):

Final Written Reports for Final Projects due 12/15/2020 at 11:59 pm.

Semester & Mini-2 Last Day of Classes = 12/11/20
Final Grades due: December 23, 2020 by 4 pm

Other Topics Not Currently Covered: