Homework 3: Evaluate Designs for Scrolling
Assigned Monday, Feb. 17, 2014, due: Monday, March 3, 2014
Be sure to see the policies for grading, late turn-in, cheating etc, on the homework policy page.
NOTE: TURN IN THIS ASSIGNMENT IN HARDCOPY IN CLASS on Monday, March 3, 2014. Please print out your entire report on paper, staple it together, and bring it to class. (If you have to miss class for any reason, you can turn in your hardcopy in advance in my office, NSH 3517, or if necessary, upload your assignment to Blackboard to demonstrate that you have it done on time.) Be sure your name is on your report!
In this assignment, each student will evaluate four (4) different designs for scrolling using two (2) different people, and write up a report on the results. Note that this assignment is quite similar to Homework 1, but we have increased the number of technologies and decreased the number of people you need to do.
Scrolling Techniques:
Each student will choose four (4) different scrolling techniques. We hope that we can get good coverage of all of the kinds of scrolling techniques that are currently in use, with approximately the same number of people doing each one. If you have an "interesting" way of doing scrolling, please use that as one of your choices. The assignment of people to techniques will be in this GoogleDoc.
For example, here are some scrolling techniques we thought of that you can pick from:
- A regular scroll bar on Windows, using only the arrow buttons on the end.
- A regular scroll bar on Windows, using only dragging of the "thumb".
- A regular scroll bar on Windows, letting the user use it however they want.
- A mouse wheel with "notches" so the user can feel the increments of scrolling
- A mouse wheel with no "notches" so scrolling is smooth
- Using a mouse wheel or the pointing stick under windows where you press down and drag up and down to scroll (a "rate controlled" scroll)
- The 2-finger scroll on a Macintosh
- Scrolling on a desktop machine using the keyboard arrow keys (optionally, along with the "PgUp" and "PgDn" keys -- at least for Windows)
- A "hand" to scroll by dragging the contents, like is available in many graphics programs and Adobe Acrobat
- The iPhone "flick-scroll"
- The iPad "flick-scroll"
- Scrolling on an Android Smartphone
- Scrolling on an Android tablet
- Scrolling on a non-touch-screen tablet like a eReader that isn't touch sensitive, using whatever kind of scrolling it supports.
- etc. -- what other scrolling techniques can you get access to?
Extra points on this assignment for doing a fifth or more techniques!
Everyone enter the techniques you will do here.
Participants
Every student should test two (2) people:
- Yourself -- you should use the four scrolling techniques and time yourself.
- Someone who is not like you - for example, someone your parent's age, a child, someone with little or no computer skills, etc. It is OK to use the same person you used for "someone not like you" for Homework 1.
Extra points on this assignment for testing a third or more people!
Test Software
The professor and TA are creating a version of scrolling test software for the web. You will be able to run the software from any web-enabled device. The software by default will display pages full of content, and users will need to scroll down to find a link, and tap on the link. The page will then record the times. For your two people, use different orders. (If you do extra techniques or extra people, then still have everyone do all the techniques, with different orders for the different people.) Note that it only takes a little over a minute to run each person for one technique through the whole test.
Run the software from here: http://www.cs.cmu.edu/~bam/uicourse/2014inter/ScrollExperiment/
(Please let the Professor know of any bugs or issues with the software.)
Make sure the participants read the instructions on the first few pages, so they know where to click -- the links are (purposely) not obvious.
Be sure to run your device so the longer dimension is horizontal. That is, it should be wider than it is tall, also called "Landscape" orientation--sideways for phones. If you are using a regular computer, make the browser window be so that at least a little scrolling is needed for all of the pages of the experiment -- you should go through the experiment once to make sure.
When the test is done, the software will display the results, which you should copy and paste into the class's GoogleDoc form, and also use the results in your own analysis for your report.
Paste your results into this form on GoogleDoc.
Preference and Demographic Questionnaire
You should create a simple (paper) questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see below), experience with computers and experience with the devices and scrolling techniques you are using. Then, you should have question(s) about their preferences and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.
Report
The deliverable for this homework is a written report. We expect it to be maybe about 3-4 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Print this report and turn it in before class on the due data. Be sure your name is on your report! The report should have the following sections:
- A detailed description of your four
scrolling techniques. This should be sufficient to allow the reader to reproduce your test exactly. For example, if doing a scroll bar with a mouse on a PC, what is the screen size and what kind of mouse were you using? What mouse settings did you use on the computer? What browser did you use to run the test? If doing a Smartphone, what exact screen resolution and screen size (diagonal)? (For example, you can often find these by searching Google for "screen resolution" -- e.g., Apple says the "Display" on the iPhone 5s is: "4-inch (diagonal) Retina display /
1136-by-640 resolution /
326 ppi"). It would also be great to include screenshots or photos of your scrolling techniques.
- A detailed description of your users. (Note: not their names!) Describe all the relevant demographic characteristics, like their experience with computers, their experience with scrolling techniques you used, whether they are left-handed or right-handed (and if left-handed, did they use their left hands -- many left-handed people still use computers right-handed), etc.
- A detailed description of the environment in which you did the test. In particular, were the subjects seated at a desk, standing, at a table? If a phone or tablet, was it on the table or in their hands? (You could even test both if you want and see if there is a difference.) Also include the order of the techniques used for each user and explain why you chose that order.
- Then present your results:
- Present the actual measured results for the total time required for each of your scrolling techniques for your users and the averages for each technique across the users, etc.
- Note that the individual scroll times are not that interesting -- use the total time that the person took with that technique. Which technique was fastest?
- If you are interested, you can also see if there are differences for short scrolls vs. long scrolls -- that is, do your comparison of techniques separately for the total time for all short scrolls, for all long scrolls and for all scrolls (total). For example, pages 13,14,15,16 are designed to be short scrolls, and 17,18,19,20 are long scrolls.
- If you are interested, you can also see if there are differences for unknown scrolls vs. known scrolls -- that is, do your comparison of techniques separately for the total time for the first part, when the user did not know how far to go, versus the second part when the user did know how far they would have to go. Are there any differences? Did users try different strategies for longer scrolls vs. shorter scrolls when they knew how far they had to go?
- Which seemed the most accurate? (Note that this test does not measure accuracy, so you will have to estimate accuracy from observations. Did users tend to overshoot when scrolling? Did they tap in the wrong place?)
- Which did the subjects prefer and why?
- Was there any effect of the order in which people used the techniques (did it seem like there was a learning effect happening)? Since participants will be doing the test four times, you might expect that they might begin to remember where to scroll to on the "unknown" pages after a while. Did this seem to be happening?
- Next in your report, have a discussion section that shows that you have command of the following topics from the class syllabus, with respect to the scrolling techniques you studied:
- The design issues regarding the different scrolling techniques, especially the trade-offs among them.
- What did your tests show about the performance? Were the tests you used the right ones? What did the tests miss (not measure)?
- Describe the historical progression of scrolling techniques and the factors that impacted their evolution and eventual widespread adoption. In particular, compare and contrast your results with the results reported in the following two references for other scrolling techniques:
- William Buxton and Brad Myers. "A Study in Two-Handed Input," Proceedings SIGCHI '86: Human Factors in Computing Systems. Boston, MA.
April 13-17, 1986. pp. 321-326. pdf or html and video.
- Brad Myers, Kin Pou ("Leo") Lie and Bo-Chieh ("Jerry") Yang, "Two-Handed
Input Using a PDA and a Mouse", Proceedings CHI'2000: Human Factors in
Computing Systems. April 1-6, 2000. The Hague, The Netherlands. pp. 41-48. local pdf.
- Finally, you can optionally have section of future work, for example, if you have any ideas for new studies, or improved ways of doing scrolling.
- Include a blank copy of your questionnaire (just like it was given to the participants) in an appendix.