Homework #1: Evaluate the Usability of a User Interface Tool or Toolkit
10% of grade. Tuesday, Jan. 15, 2013 - Tuesday, Jan. 22, 2013 (due at 12 noon!!)
List of tool choices from class are in this GoogleDoc.
(Email the professor if you want to change, as long as you pick a tool no-one else has picked).
Here is a file of the students' presentations: PDF.
User Interface tools are generally designed to be used by programmers, but
programmers are people too! Pick a user interface tool or toolkit. It doesn't
have to be an interactive tool -- you can evaluate a library or framework. For
example, so candidate tools are things like: the Java language, Swing, Microsoft Visual Basic, Apple xCode, Adobe Flash, etc. There is a preliminary example list of such tools here as a GoogleDoc. In order to help fill out this list, the first part of this homework is for each person to do the following (this part is due with the rest of the homework on Jan 22):
- Find at least 3 (three) new tools that are in active use to add to the List of Programming Languages and Tools. (You must find tools that are not currently directly in the list. However, it is OK if a tool is in a list referenced from that page.) For each tool, include:
- The tool name.
- A URL for more information about the tool, or where to buy or download it.
- What type of tool is it? (Programming Language, SDK, IDE, builder, etc.).
- What category of the list should the tool be filed under?
- One sentence description of the tool.
Separately, everyone in the class must pick a different toolkit to evaluate
(see list of possibilities and list of what people selected).
Note: it is fine to evaluate a tool already on the list, as long as it is a tool no-one else has selected. Some tools have many parts, so it is OK if different people do different parts
of the same tool. For example, someone could do the Java language,
while someone else does the Swing toolkit for Java, while someone else does
the Eclipse IDE for Java, etc. We will assign tools during the first lecture on Tuesday, Jan. 15, 2013 during class, so be prepared with which tool you would like to do (note that you might not get your first choice, since everyone must do a different tool).
You will evaluate your tool with either
Neilsen's Ten Usability
T.R.G. Green's Cognitive
Dimensions Framework. (It would alternatively be OK to do an actual user study,
like Jeff Stylos's, but I don't think there is time!) Your report should discuss both problems
and things done well. You should include pictures or code or
documentation snippets that illustrate each of your points. Each point should be annotated with
the particular guidelines or cognitive dimensions that are being violated or being followed. If
violated, you might suggest a fix. Discuss at least 10
points that illustrate at least 5 different heuristic
guidelines or at least 5 different cognitive dimensions. I am looking for insight, subtlety and depth (thus
saying "the system doesn't have good color choices because it doesn't use color" is pretty trivial and won't count for
- Prepare a written report that explains the tool and your evaluation of it.
The report should be understandable to someone who is a programmer, but not familiar with your particular tool.
- In class on Tuesday, Jan. 22, 2013 or Thursday,
Jan. 24, 2013, each person will also give an five (5)
minute presentation about your findings. This should include a brief
overview of your tool for those who don't know it, what you found about good
and bad aspects of the tool, and any observations on the evaluation
technique (CD or HE) as a way to understand whether the tool is good or not.
Please prepare slides to show to the class. Please keep to 5 minutes
so we can fit everyone in (that means practice). (Obviously, you won't have time to discuss all 10 points you find -- just talk about the most interesting.) Your presentation
will affect your grade on this homework. Typically, you should plan to have 3 slides with text on them: 1 slide about the tool, 1 slide about good features, and 1 slide about bad features, plus as many slides as you can fit with pictures on them to illustrate your points. To make the transitions between people go quickly (and so people presenting on the second day don't get to continue working), all slideshows must be turned in as PDFs, and we will run them from a single computer. You will not be able to do a live demo of your tool.
Turn in a hardcopy of your report and slides in class on Tuesday, Jan. 22, 2013, and also upload your report and slides (as a ZIP file containing two (2) PDFs--one of your report and one of your slides)
to blackboard before class. (See the instructions about turn-ins and the late policy.) **PLEASE UPLOAD YOUR ZIP FILE BEFORE 12:00 noon on Tuesday, Jan. 22, 2013, so I have time to put them all together before class**.
If you are not familiar with Heuristic Analysis, read about it in the
following resources. Note that the list of heuristics on the web site are slightly
different than those in Nielsen's Text. To avoid confusion, please use only the ones that
lists on his web site.
- Nielsen Text: Chapter 5
- Jakob Nielsen, Heuristic Evaluation. On
line in html format. Includes the list
of 10 heuristics to be used in this assignment.
- Jakob Nielsen, "Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation
Barrier", Cost-Justifying Usability, edited by Randolph G. Bias and Deborah J.
line in html format.
- Rolf Molich and Jakob Nielsen. "Improving a human-computer dialogue",
Communications of the ACM, March 1990. Volume 33 Issue 3. pp. 338 - 348.
ACM DL PDF
- Jakob Nielsen and Rolf Molich. "Heuristic evaluation of user interfaces,"
Proceedings CHI'90, Human factors in computing systems, 1990. Seattle, Washington
ACM DL PDF
If you are not familiar with Cognitive Dimensions Analysis, here are
- T.R.G. Green, Cognitive Dimensions of Notations.
Dimensions of Notations Resource Site
- Green, T.R.G. and Petre, M., “Usability Analysis of
Visual Programming Environments: A 'Cognitive Dimensions' Framework.”
Journal of Visual Languages and Computing, 1996. 7(2): pp.
ScienceDirect or preprint pdf
- Clarke, Steve, “Measuring API Usability.”
Dr. Dobb's Journal; Special Windows/.NET Supplement, May, 2004. pp.
- Francesmary Modugno, T.R.G. Green and Brad A. Myers. "Visual
Programming in a Visual Domain: A Case Study of Cognitive
Dimension," Human-Computer Interaction '94, People and Computers.
Glasgow, Scotland, August, 1994. pp. 91-108.
Here is a GoogleDoc with a tentative assignment of class members to tools (see the list of possible tools to choose from)
Back to Homework Overview
Back to 05-830 main page