Homework 3: Evaluate Designs for Scrolling

Assigned: Thursday, Feb. 17, 2022; Due: Tuesday, Mar 1, 2022 at 1:25pm ET


In this assignment, each student will evaluate three (3) different designs for scrolling using three (3) different people, and write up a report on the results. Note that this assignment is quite similar to Homework 1, but we have increased the number of techniques you need to do (but it only takes <5 minutes per device). Students will enter the devices they will evaluate into a GoogleDoc, and run the Scrolling Test using our web app.

Scrolling Techniques

Each student will choose three (3) different scrolling techniques. We hope that we can get good coverage of all of the kinds of scrolling techniques that are currently in use, with approximately the same number of people doing each one. If you have an “interesting” way of doing scrolling, please use that as one of your choices. You should enter the devices you will do into this GoogleDoc. (Note that you need to be using your CMU Andrew Google account to access this.)

For example, here are some scrolling techniques we thought of that you can pick from:

Extra points (up to +3) on this assignment for doing a fourth or more techniques! Also, extra credit (up to +2) if you test a really interesting scrolling technique, like a VR controller, game controller, etc.

Everyone enter the techniques you will do here.


Every student should test three (3) people: (it is OK to use the same people you used for Homework 1):

Extra points (up to +3) on this assignment for testing a fourth or more people!

Test Software

The professor and TA are creating a version of scrolling test software for the web. This software is brand new this year! It is based on the final project from IxT 2019 by Cem Ergin, Emily Porat, Sijia Li, and Chun Wang.

You will be able to run the software from any web-enabled device. The software by default will display pages full of content, and users will need to scroll down to find the appropriate line, and get that line inside the grey box at the top. The page will then record the times. For your three people, use different orders for the scrolling techniques. Be sure the mention the orders in your report. (If you do extra techniques or extra people, then still have everyone do all the techniques, with different orders for the different people.) Note that it only takes about 5 minutes to run each person for one technique through the whole test, so it should be less than 15 minutes per person.

Make sure the participants read the instructions on the first page, so they know what to do.

The test uses Google Sheets to record the answers. Happily, we created the scrolling test app a different way from homework 1, and now Google doesn't require you to log in or to confirm that you are ok sharing your data, but it is still saved behind the scenes.

Click here to run the scrolling experiment software
(Please let the Professor and TA know of any bugs or issues with the software.)

Design of the Test

We have tried to make sure the content is always about 10 screenfuls, so lots of scrolling will be needed. The active area is square. For regular computers, please make the window full-height and width of the screen. For phones and tablets, please use portrait mode, so the longer dimension is vertical, since this is the normal way that these devices are used.

For most kinds of scrolling on a regular computer, the pointer cursor must be inside the active square scrolling region. For example, to use 2-finger scrolling on the touchpad, or the mouse scroll wheel, the user must move the cursor into the active area after clicking on the start button. It does not work to leave the cursor on the start button, or in the black or grey areas.

The "formal test" does 20 different scrolls in each of 2 phases (40 total scrolls). The target lines are the same for the 2 phases, and are the same for all participants: 11,11,20,20,30,30,40,40,50,50,60,60,70,70.80,80,90,90,99,99. However, the order is randomized for each run, so users should not be able to guess which one they are doing or what is next, etc. We start with line 11 so the target is never visible before starting, and we end with 99 (the last row). Each value is used twice.

The two phases of the test are:

  1. Somewhere on the page are a row of stars. These should be scrolled to be in the target area. This tests for the situation when the user needs to locate something by visually scanning the document.
  2. The user is told the specific line to jump to. This tests for the situation where the user knows (at least approximately) where the content they are looking for is. Examples might be scrolling an alphabetical list for a known last name, or remembering that the content you are looking for is at the top or bottom of the document. Note that the scrolling pegs at the end, so 99 should be especially easy to get to.

The target area is designed to be about 1.5 lines tall, so it should be possible to get the target line into the area. Note that if using a mouse wheel or equivalent scrolling devices, the increment may need to be adjusted to be "1 line". (My scroll wheel was originally set to 3 lines, and it kept skipping over the target area, both going up and then back down!)

Preference and Demographic Questionnaire

Same as for homework 1, you should create a simple questionnaire to give each of your subjects (including yourself!) to be filled out at the end after doing all the tasks. It should ask basic demographic information (see report below), experience with computers and experience with the scrolling techniques you are using. Then, you should have question(s) about their preferences among the scrolling techniques they used and why. You need to include a blank copy of the questionnaire with your report so we can see what you asked.


The deliverable for this homework is a written report. We expect it to be maybe about 4-5 pages (not including figures or the copy of the questionnaire), single-spaced in TimesRoman 12 point font. Create a pdf of this report, and upload it to Canvas before class on the due date.

Be sure your name is on the first page of your report and in the file name! Your pdf file should be named: lastname_firstname_ixt22_hw3.pdf. For example, mine would be: myers_brad_ixt22_hw3.pdf.

The report should have the following sections. (Note that you should use this as a template using these section names). Please put your information into the appropriate section as described below, so it is easier to grade.


The various parts of this homework will be calculated as follows, so you know how much each part is worth: