Schedule and Readings

Num Date Class Content
1 Tuesday,

Introduction to Course; What are "Software Structures for User Interfaces"; and Why are user interfaces hard to design and implement?

Please fill out this questionnaire before the first class:

Slides for Lecture 1 and pdf
Video for Lecture 1 on Canvas (only available to people in the class)

Required Readings:

  • Brad A. Myers. "Challenges of HCI Design and Implementation," ACM Interactions. vol. 1, no. 1. January, 1994. pp. 73-83. ACM DL
    Discusses why user interfaces are important, and why they are hard to design and implement.

Recommended Reading:

  • Brad A. Myers. "A Brief History of Human Computer Interaction Technology." ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44-54. ACM DL

Start on Homework 1 - html, CSS, and JavaScript

2 Thursday,

Review of HTML and CSS

Slides for Lecture 2 and pdf

Required Readings:


Development setup, Git and GitHub, HTML/CSS

Slides for all labs will be in this folder on Canvas

3 Tuesday,

Review of JavaScript

Required Readings: See list with Lecture 2

Slides for Lecture 3 and pdf

4 Thursday,

Input 1: Conventional Input Models for Handling Input Events

Slides for Lecture 4 and pdf

Required Readings:

  • The JavaScript resources listed above will cover event handling for Input Events, DOM Events, etc., but here are some extra resources:

JavaScript, DOM, and events

Course add Deadline is September 12, 2022.

5 Tuesday,

What is "User Interface Software", Overview of UI Software and Tools

Slides for Lecture 5 and pdf

Optional Readings:

  • Brad A. Myers. "Graphical User Interface Programming," chapter 48 of Computer Science Handbook -- Second Edition. Allen B. Tucker, editor in chief. Boca Raton, FL: Chapman & Hall/CRC Press, Inc., 2004. pp. 48-1 - 48-29. pdf  (This is a revision of Brad A. Myers. "User Interface Software Tools," ACM Transactions on Computer-Human Interaction. vol. 2, no. 1, March, 1995. pp. 64-103. ACM DL Reference)
  • Brad A. Myers, Scott Hudson and Randy Pausch. "Past Present and Future of User Interface Software Tools," ACM Transactions on Computer Human Interaction, (Mar. 2000). Volume 7, Issue 1. pp. 3-28. ACM DL Reference.

Turn in Homework 1.
Start on Homework 2 - Input Handling

6 Thursday,

Output 1: Basic 2D Computer Graphics

Slides for Lectures 6 & 7 and pdf

Required Readings:


Advanced Input Event Handling

7 Tuesday,

Output 1: Basic 2D Computer Graphics, continued

8 Thursday,

Input 2: Declarative input models; "Interactor" (Behavior) Objects in Garnet and Amulet
Output 2: Graphics Models: Structured Graphics; Multiple Hierarchies; Object-Oriented Techniques

Slides for Lecture 8 and pdf

Recommended Readings:

  • Brad A. Myers. "A New Model for Handling Input," ACM Transactions on Information Systems, vol. 8, No. 3. July, 1990. pp. 289-320. ACM DL Reference.
  • Brad A. Myers, Richard G. McDaniel, Robert C. Miller, Alan Ferrency,  Andrew Faulring, Bruce D. Kyle, Andrew Mickish, Alex Klimovitski, and Patrick Doane. "The Amulet Environment: New Models for Effective User Interface Software Development", IEEE Transactions on Software Engineering, Vol. 23, no. 6. June, 1997. pp. 347-365. local pdf or IEEE DL.

Optional Readings:

  • David S. Kosbie, Brad Vander Zanden, Brad A. Myers, Dario Giuse. "Automatic Graphical Output Management", in Brad A. Myers, editor. The Garnet Compendium: Collected Papers, 1989-1990. Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-90-154, August, 1990. pp. 30-43. pdf.

Graphics 1

9 Tuesday,

UI Software Models: Lexical-Syntax-Semantics, Seeheim Model, MVC, Design Patterns, State Diagrams

Slides for Lecture 9 and pdf

Recommended Readings:

  • Glenn Krasner and Stephen T. Pope, "A Cookbook for Using the Model-View-Controller User Interface Paradigm in Smalltalk-80", Journal of Object-Oriented Programming (JOOP). August-September, 1988. vol. 1, no. 3. pp. 26-49. pdf scan at UCI

Optional Readings:

  • Mark Green, "A Survey of Three Dialogue Models," ACM Transactions on Graphics (5)3, Jul, 1986, pp 244-275. ACM DL Reference
  • William Buxton, "Lexical and Pragmatic Considerations of Input Structures," Computer Graphics, January, 1983, (17)1, pp. 31-37. (lexical, syntactic, semantic, etc.). html
  • chapter 10 from Olsen's book: "Editing Dialog Models". User Interface Management Systems: Models and Algorithms.Morgan Kaufmann; (November 1991) (pdf of the chapter (CMU only), also avaialble in Special Course Readings on Canvas), or Amazon listing for book.

Turn in Homework 2.
Start on Homework 3 - Graphics using SVG and Canvas

10 Thursday,

UI Implementation Concepts: Callbacks, Resources, Widget Hierarchies, Geometry Management

Slides for Lecture 10 and pdf

Optional Readings:


Graphics 2

11 Tuesday,

React for JavaScript, 1

Slides for Lectures 11 & 12 and pdf

Required Readings:

Optional Readings:

12 Thursday,

React for JavaScript, 2

Same slides as lecture 11, but reload for new content!


React 1

13 Tuesday,

Evaluation of APIs and UI Tools, API Usability, Cognitive Dimensions

Slides for Lecture 13 and pdf

Recommended Readings:

  • Brad A. Myers, A. J. Ko, Thomas D. LaToza, and YoungSeok Yoon. "Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools," IEEE Computer, Special issue on UI Design, 49, issue 7, July, 2016, pp. 44-52. IEEE DL or local pdf.
  • Brad A. Myers and Jeffrey Stylos, "Improving API Usability", Communications of the ACM, vol 59, No. 6, June, 2016, pp. 62-69, ACM DLhtml or local pdf.
  • Jakob Nielsen, Heuristic Evaluation. On line in html format. Includes list of 10 heuristics.
  • Green, T.R.G. and Petre, M., “Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework.Journal of Visual Languages and Computing, 1996. 7(2): pp. 131-174. ScienceDirect.
    Includes an analysis of HyperCard

Optional Readings:

  • Jeffrey Stylos and Brad Myers, "Mapping the Space of API Design Decisions," 2007 IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC'07. Sept 23-27, 2007, Coeur d'Alene, Idaho. pp. 50-57. ieee or local pdf
  • Brian Ellis, Jeffrey Stylos, and Brad Myers. "The Factory Pattern in API Design: A Usability Evaluation". International Conference on Software Engineering (ICSE'2007). May 20-26, 2007. Minneapolis, MN. pp. 302-312. ACM DL or local pdf
  • T.R.G. Green, Cognitive Dimensions of Notations. Cognitive Dimensions of Notations Resource Site

Turn in Homework 3.
Start on Homework 4 - Using React

14 Thursday,

Command Objects & Support for Undo

Slides for Lecture 14 and pdf version

Recommended Readings:

  • Brad A. Myers and David Kosbie. "Reusable Hierarchical Command Objects," Proceedings CHI'96: Human Factors in Computing Systems. Vancouver, BC, Canada. April 14-18, 1996. ACM DL Reference

Optional Readings:

  • Command Objects in Amulet (note: only the overall design is relevant, not the details)
  • Brad A. Myers. "Scripting Graphical Applications by Demonstration," Proceedings CHI'98: Human Factors in Computing Systems. Los Angeles, CA, April 18-23, 1998. pp. 534-541. ACM DL Reference and local pdf and video. (Topaz)

Take-home Midterm Exam released Wednesday and due Friday before class


React 2

X Tuesday,

CMU Mid-Semester Break - no classes

X Thursday,

CMU Mid-Semester Break - no classes


CMU Mid-Semester Break - no labs

Mid Semester Grades due: October 24, 2022

15 Tuesday,

Connecting to the backend - Web Services and Databases

Slides for Lecture 15 & 16 and pdf version

Required Readings:

Please fill out the midterm survey:
Turn in Homework 4 (deadline extended until Friday, 10/28/22).
Start on Homework 5 - Implement Undo using Command Objects

16 Thursday,

Connecting to the backend - Web Services and Databases, continued

Same slides as Lecture 15 and pdf


Tartan Community Day; No Classes

17 Tuesday,

Brad away at Dagstuhl

Some Other Web Toolkits - jQuery, AngularJS, Vue, Bootstrap, Svelte, etc.
presented by Alex Cabrera

Slides for Lecture 17 (pdf)

Recommended Readings:

18 Thursday,

Brad away at Dagstuhl

Designing Large Web Applications
presented by Alex Cabrera

Slides for Lecture 18 (pdf)

Recommended Readings:

Turn in Homework 5 (deadline extended until 11/8)
Start on Homework 6 - Connecting to a Backend


Using web services & Remote Databases

Course Drop Deadline is November 7, 2022.

19 Tuesday,

Toolkits for building speech/conversational/chatbot User Interfaces, and Visualizations

  • Speech Examples: DialogFlow, Microsoft Bot Network, Voiceflow, ...
  • Viz Examples: Prefuse, Flare, Improvise, VTK, Excel, Many Eyes, Google Charts, Tableau, VizQL, ggplot2, Protovis, D3

Slides for Lecture 20 and pdf version

Recommended Readings:

Optional Readings:

20 Thursday,

Toolkit support for Gestural Input Techniques

Slides for Lecture 20 and pdf version

Required Readings:

  • J.O. Wobbrock, AD Wilson, Y Li. "Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes", In UIST '07, p. 159-168. ACM DL Ref

Optional Readings:


Data Visualization 1

21 Tuesday,

Software Engineering for UIs: How to create high-quality UIs in the context of Agile and other software development processes

Slides for Lecture 21 and pdf version

Recommended Readings:

  • Budwig, M., Jeong, S. and Kelkar, K. When user experience met agile: a case study. In Proceedings of CHI'2009: The 27th international conference extended abstracts on Human factors in computing systems (Boston, MA, USA, 2009). ACM, 3075-3084. ACM DL PDF

Turn in Homework 6 (deadline extended until 11/17)
Start on Final Project!

22 Thursday,

Simple User Interface Toolkits and End-User Programming for UIs; Low-Code / NoCode

  • Examples: Scratch, Alice, AirTable,, Zapier, LabView, IFTTT, (older: Yahoo Pipes)

Slides for Lecture 22 and pdf version

Recommended Readings:

  • Conway, M., Audia, S., Burnette, T., Cosgrove, D., Christiansen, K., Deline, R., et al. (2000, Apr 1-6). Alice: Lessons Learned from Building a 3D System For Novices. Paper presented at the Proceedings of CHI 2000, The Hague, The Netherlands. ACM DL Ref, or citeseer ref
  • See also links in the lecture slides for many more papers and videos about these systems!

Optional Readings:

  • Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344. ACM DL Ref, or citeseer ref
  • J.F. Pane, B.A. Myers, and L.B. Miller, "Using HCI Techniques to Design a More Usable Programming System," 2002 IEEE Symposia on Human Centric Computing Languages and Environments (HCC'2002). Arlington, VA, September 3-6, 2002. pp. 198-206. local pdf. (Winner Most Influential Paper Award for important influences on VL/HCC research or commerce over the last 10+/-1 years in 2012.)
  • Brad A. Myers, John F. Pane and Andy Ko, "Natural Programming Languages and Environments". Communications of the ACM. 47(9), pp. 47-52. ACM DL Ref

Turn in Homework 6 (extended deadline)

LAB 10

Data Visualization 2

23 Tuesday,

Toolkit support for Assistive and Accessible Interfaces; Web Accessibility

Slides for Lecture 23 and pdf version

Recommended Readings:

  • W3C, "Introduction to Web Accessibility," on-line
  • Zimmermann, G., Vanderheiden, G. Accessible design and testing in the application development process: considerations for an integrated approach. Univ Access Inf Soc 7, 117–128 (2008).

Optional Readings:

Form final project groups

X Thursday,




24 Tuesday,

Toolkits for 3D Programming and the UIs of Games

Slides for Lecture 24 and pdf version

Optional Readings:

  • Richard Fine, "Going deep with IMGUI and Editor Customization", December 22, 2015, blog post -- discusses the event loop for Unity's old immediate-mode GUI system
  • J. Blow, “Game development: Harder Than You Think”, ACM Queue, ACM, New York, 2004, pp. 29-37. ACM DL REF
  • Take a look at standard game engines of today: Unity Game EngineUnreal Engine, and (for non-programmers) GameMaker. There are dozens of other game engines available.
  • OpenGL is the graphics library standard, and it is open source. However, it is highly complex. For a gentler introduction and some videos demonstrating its capability, look at a javascript-based wrapper for OpenGL called three.js.

One-page proposal for Projects due on Wednesday, 11/30/22 at 3:05pm on Canvas

25 Thursday,

Interactive Tools: Prototypers (HyperCard, Director, Visual Basic, Balsamiq), Interface Builders, Sketching Tools

Slides for Lecture 25 and pdf version

Recommended Readings:

  • James Landay and Brad A. Myers. "Interactive Sketching for the Early Stages of User Interface Design," Proceedings CHI'95: Human Factors in Computing Systems. Denver, CO. May, 1995. pp. 43-50. (html) or ACM DL Ref

Optional Readings:

  • Brad A. Myers. "Separating Application Code from Toolkits:  Eliminating the Spaghetti of Call-Backs,"  ACM Symposium on User Interface Software and Technology: UIST'91, Hilton Head, SC, Nov. 11-13, 1991. pp. 211-220. ACM DL Reference
  • Walker, M., Takayama, L., and Landay, J. (2002) High-fidelity or low-fidelity, paper or computer medium? Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (in press). 
  • James A. Landay and Brad A. Myers. Just Draw It! Programming by Sketching Storyboards.   Carnegie Mellon University School of Computer Science Technical Report, no. CMU-CS-95-199 and Human Computer Interaction Institute Technical Report CMU-HCII-95-106. November, 1995. html or pdf
LAB 11

3D programming

Three.JS and basic 3D rendering in a browser

26 Tuesday,

Constraints and Data Bindings

Slides for Lecture 26 and pdf version

Recommended Readings:

  • Scott E. Hudson. "Incremental attribute evaluation: a flexible algorithm for lazy update," ACM Transactions on Programming Languages and Systems, Volume 13 , Issue 3 (1991), Pages 315-341.  ACM DL Reference

Optional Readings:

  • Brad Vander Zanden, Brad A. Myers, Dario Giuse and Pedro Szekely. "Integrating Pointer Variables into One-Way Constraint Models," ACM Transactions on Computer-Human Interaction. vol. 1, no. 2, June, 1994. pp. 161-213. ACM DL Reference
  • Brad Vander Zanden, "An Incremental Algorithm for Satisfying Hierarchies of Multi-way, Dataflow Constraints", ACM Transactions on Programming Languages and Systems, 18(1), January, 1996. pp. 30-72. ACM DL Reference
  • Bjorn N. Freeman-Benson, John Maloney and Alan Borning. "An incremental constraint solver," Communications of the ACM . Volume 33 , Issue 1 (1990). Pages 54-63. ACM DL Reference

Please fill out the class survey: and the official CMU survey and the peer survey about your project (required):

27 Thursday,

Presentation of Final Projects


1. Vacation Group
2. Blokus
3. 4102_Quartet
4. Chess Web App
5. Mixtapify
LAB 12

Presentation of Final Projects

6. InfoChat
7. Sequence Game
8. EaseAI

Semester & Mini-2 Last Day of Classes = Friday, December 9, 2022

X Exam week

Final Written Reports for Final Projects due Monday, 12/12/2022 at 11:59pm.

Final Test: Start Tuesday, 12/13/2022 at 12:01am ET; Due Wednesday, 12/14/2022 at 11:59pm ET (all day Tuesday and Wednesday)

Final Grades due: December 21, 2022 by 4 pm

Other Topics Not Currently Covered: