|
home | projects | hobbies | resume | links | contact |
|
PalmPilot III Redesign |
||||
| |
||||
|
Part 1: Executive Summary The
theme of our most significant redesign for Palm Pilot III is
the connection of related data between the various
applications of the Palm Pilot. One of the main problems we
observed in our analysis of the previous product was the
user’s inability to link information in various applications
which corresponded to a single event. We recommend two
possible redesign solutions: the PalmProject, in which a
project can be used to collect all the information for a given
event, and the PalmLink, in which each item within an
application is linked directly with an item or items in
another application. We also found less significant problems,
such as a non-intuitive structure of the Datebook, difficulty
in using of graffiti successfully, buried functionality, and
lack of clear labeling for some functions. Redesigns were
suggested for these as well. The methods we used to analyze
the Palm Pilot were Contextual Inquiry and Design, Heuristic
Evaluation, Cognitive Walkthrough, and Think Aloud Protocol.
We have also proposed an experiment comparing our two major
redesign solutions. Part 2: Redesign The four primary applications of the Palm Pilot (DateBook, To-do List, Memo pad, Address Book) often contain highly interrelated information. All of these applications run simultaneously. However, only one application may be accessed from the input screen at a time; multiple applications cannot be viewed simultaneously. In the videotapes we watched of the think-aloud protocol, the user’s most common struggle was in relating information between one application and another (See Appendix 1 for an enumerated list of all of the usability problems we found)(Problems 12, 14, 19, 20, 21, 24, 25, 67) For instance, the user had a meeting of the Publications Committee scheduled for the future. This meeting had associated with it 1) travel arrangements, 2) accommodations, 3) meeting times and dates, and 4) reminders. The user’s task was to enter this information into the corresponding applications of the Palm Pilot, which she felt were 1) the Memo Pad 2) the Address Book 3) the DateBook and 4) the To-Do List respectively. The user then needed a way to associate the pieces of information in various applications to each other so that she could easily retrieve all of the information relevant to the Publications Committee meeting at a later date. Because there was no way of directly linking the information in the various applications, the user developed the clumsy work-around of entering an identical text string “Publications Committee” into each location where data about the particular meeting was entered. In this manner, at least she would be able to locate the meeting information by entering the search term “Publications Committee” into the “Find” function and then going to each of the search results individually to unearth an aspect of the meeting details. As seen in this example, the user must often perform circuitous or indirect actions in order to effect a link between two related pieces of data. Because this user is unable to see all the applications at one time, she must retrieve each bit of information about the event serially. Each time the user opens a new entry with the reference “Publications Committee” in it, she must keep adding this new information to the cumulative total of the information she had accessed before it, rather than being able to see it all together. This model is very inefficient, as it either forces the user to increase her memory load, or to write things down on scraps of paper-either of these alternatives would not be taking full advantage of the Palm Pilot as a personal organizer. In
addition to the evidence from the think-aloud, one of the
primary problems that arose from our Contextual Inquiry
analysis was the disorganization associated with numerous
memory aids being utilized by the user. This is analogous to
the problem of information being dispersed throughout the Palm
Pilot’s application with no connective mechanism. The user
had duplicate records between her PC and wizard, which
demonstrated a lack of coherence between these two memory aids
(71). In addition, the user had difficulty finding entries she
made in a paper notebook (74), and she could not remember
where email had been stored (73).
These breakdowns could all be remedied if the user had
one place to store pointers to or copies of all of her
information, and if that information was interconnected.
These
problems suggest that a new interaction idiom closer to actual
user practice is needed.
One possibility is a project-centered interaction,
which we call the “PalmProject”.
Much like the folders on a desktop PC, the primary
object of interaction would be a project document.
Each project would be used store information about a
particular event; for example, all information about a
particular meeting could be stored in a project, as shown in
Appendix 2. Inside each project there would exist lines
labeled with icons for the Datebook, Memo Pad, Address Book,
or To-Do List, and the user would enter the data regarding the
project on one of these lines, according to where the
information belonged. The information would then automatically
be passed into the corresponding application as well. For
instance, in our user’s example, she could have simply
created a new project termed ‘Publications Committee
Meeting’. The
meeting could then have associated dates of travel,
accommodation address and reservation/phone #’s, and meeting
times. Each of
these items would be entered by tapping on the appropriate
application icon from within the project view.
A window within the project view would then appear, and
this window would ‘host’ the appropriate application for
entering that type of data. For example, if the user tapped on
the ‘Datebook’ icon within the project view in order to
enter some date information, the Datebook application would
appear within a window inside of the project application. In
addition, each project would compare itself to others to look
for scheduling conflicts.
The user could then choose to associate the conflicting
data with the current project (for instance, a phone call that
needed to be made on the same day as one in the current
project but was linked with a different project could be
associated with the current project and so be removed from the
conflict list and added to the current project view). The user
could either access the information from within the individual
projects, or from the applications where the corresponding
information was stored. The
precedents for this idiom are numerous, one of the most
interesting being the Squeak desktop interface developed at
Xerox PARC. In Squeak, each project is saved as a separate desktop so
that all relevant documents and applications are gathered
together in place. When
working on that project, everything needed stays on the
desktop. When leaving that project, that desktop is minimized
and saved, with all relevant data staying just as it was when
you left it. Appendix 2 provides a detailed description of a
typical task performed with the proposed project view.
An
alternative solution to the project metaphor would be to
simply link data between applications, a solution we call the
“PalmLink”. This hypertext style of interaction would
allow the user to tap on a link and immediately go to the
linked data. This
style of interaction would have the benefit of not requiring
the user to create an entire project for simple data
connections, in which an event is comprised of information in
only one or two applications. Appendix 3 illustrates a sample
task of connecting data using this technique.
There are several disadvantages to this approach,
however, that led us to suggest the project metaphor as the
primary redesign. Hypertext
style links work well when linking two pieces of information.
With more links, however, it becomes necessary to
either access them sequentially or to indicate each link with
a separate icon. The
sequential access becomes increasingly difficult and confusing
as more data is connected.
Adding icons for each link also becomes problematic,
however, because of the limited screen real estate on the Palm
Pilot. The
experiment proposed in Part 3 is geared towards determining
which of these data connection styles is more effective for
the user.
Additional Redesign Ideas
Appendix
4 lists the major categories of problems that we feel need to
be addressed through redesigns.
The table lists a short description of the problem as
well as the corresponding number from Appendix 1 – the big
table of
problems. The
primary redesign ideas for a data connection mechanism (PalmProject
and PalmLink) is listed first, with less significant redesign
ideas and problems following.
The following paragraphs describe each of the less
significant redesign ideas in more detail.
An
important change we decided to make was to restructure
the DateBook (5, 6, 8, 30, 49, 58). From the Think-Aloud,
we found that the user had difficulty with the organization of
months, days, and weeks. She needed to go from December to
November and did not understand how, as she began in the Day
view. We believe that the user’s mental model is to think in
terms of the bigger picture (months) when scheduling
appointments, rather than days and weeks. Our proposed
redesign is to have the month view as the default Datebook
view and have the individual day selectable from the month
view. Based
on our Heuristic Evaluation, we found some significant redesigns for grafiti (26, 48). First, we suggested that the number
pad and letter pad be separated using a hardware division, as,
this distinction is not at all clear to the user. Second, we
feel that immediate feedback is necessary to the user about
the strokes he or she makes on the grafitti pad. The Pad
should allow grafitti feedback while you write Based
on Heuristic Evaluation we found that
critical standard functionality ( 28, 32, 54, 59, 64) such
as Adding, Deleting, Details and Notes is often buried.
Manipulation of (Adding, Deleting, Details, Notes) entries
should be at top-level of all applications.
Making this functionality, as well as ‘Help’, more
accessible would eliminate many of the usability problems. We
also decided that applications and functions
should be clearly labeled (3,7, 21, 29, 47). Features such
as Priorities could be labeled more clearly.
The names of individual applications such as DateBook,
which confused the user in the think-aloud, should also be
displayed in a prominent location. Also there are some cases
where pictorial information should be substituted with words.
Times (3:30-4:30), and not bars should be in DateBook
month/week views, and an icon labeled CAPS LOCK, not the up
arrow should be visible when the user wants to enter
capitalized text. Part 3: Experiment The
think-aloud data we previously discussed revealed that the
user had problems associating related data (dates and times,
location, and contact information) for a single complex event
because each type of data needed to be stored in a separate
application. Our general redesign solution was to give the
user a way to connect the related data in the various
applications so that it will be easier to retrieve all the
data simultaneously at a later date. We arrived at two
possible approaches to this solution, the “project”
approach (PalmProject) and the “hyperlink” approach (PalmLink),
as described earlier. The
main objective of our experiment is to compare these two
approaches in terms of user performance.
Our reason for choosing to do an experiment rather than
employing analytical usability methods is so that we can
quantitatively compare results for the same tasks on the two
interfaces based on concrete measures such as errors and time,
which we feel are representative measures of learnability and
efficiency. Our claim is that an interface where information
can be stored in one central location, such as in the
PalmProject, will be much more intuitive than creating links
between information in disparate applications.
When a user thinks of a particular event, as seen in
the think-aloud, he or she wants to tie all the information
for that event (names, dates, times, contact information,
notes) together, rather than to categorize it by the type of
event it is. A user can store all the information for a single
event in one place, rather than having to go to each
application separately, enter the information, and create
individual links between them. We
also feel that PalmProject will be easier to learn, as it
supports the data-driven model most operating systems claim to
use, in which one data file is created and then manipulated by
various applications. By comparing the interfaces on the
measures of time and errors, we will attempt to confirm our
claim that the project-oriented interface is more efficient
and easy to use than linking related information for a single
event. The outcome we expect is that users will clearly
understand PalmProject, and that they will be able to perform
the tasks on this interface in less time and with fewer errors
than on PalmLink. If this is the case, then the design
implication is clear: the project metaphor is clearly aligned
with the user’s mental model, and that it should be
implemented. If the users perform better on PalmLink, this
might imply that a centralized location for information is not
necessary, and links can be made from within each application.
In the case that users have about the same performance
on both measures, we may consider a compromise between the two
interfaces; at this point, further experiments may need to be
conducted to discover which features of each interface should
be carried into the redesign.
The participants in the experiment will be of two
experience levels. The first group will have no experience
with the Palm Pilot (and therefore no experience with
graffiti), but will have experience with PC conventions. The
second group will also have experience with PC conventions,
but they will have used the Palm Pilot for at least a year,
understand its various applications, and know how to use
graffiti. We chose users that have experience with at least
one other operating system because we assume that the typical
owners of small PDAs are familiar with personal computers, and
so they will be familiar with the use of such tools as file
menus, buttons, and scroll bars. In testing a specific feature
of the Palm Pilot, lack of familiarity with computing systems
is irrelevant and will only confound the data. We will use a
total of sixty subjects. We chose such a large number because
(as will be discussed later) we will have a between subjects
design with two treatment conditions (PalmProject or PalmLink)
and we will be dividing our subjects’ data by experience
level (experienced or novice Palm Pilot users). Therefore we
will have 4 experimental conditions (PalmProject, experienced;
PalmLink, experienced; PalmProject, novice; PalmLink, novice).
Our
specific hypothesis is that performance will be better on the
PalmProject than the current for both groups of subjects.
However, we hypothesize that for both the current and proposed
interfaces the existing users may make fewer errors in data
entry and general navigation, and finish the tasks in less
time, due to their experience with the Palm Pilot in general.
Existing users will have familiarity with the various
applications such as the DateBook, To-Do List, Memo Pad and
Address Book, as well as with the graffiti pad. The
experiment will be between subjects. Although users will
perform the same tasks, we will show them either PalmProject
or PalmLink, but not both. The reason for this design is to
prevent any learning effects that may take place from the
alternate interface. There are two independent variables in the experiment.
The first will be the experimental treatment: PalmProject
versus PalmLink. The second will be the subject
classification: current Palm Pilot users (experts) as opposed
to users new to the Palm Pilot (novices). We
will use two tasks in our experiment, and these tasks will be
done on both interfaces (Described in detail in Appendices 3
and 4). The first task will require the user to enter
information about a trip for someone else -this will include
dates and times, the address and phone number of a contact
person at a hotel where he or she will stay, things to do once
he or she arrives at the hotel, and notes on the places
friends have recommended that he/she visit. The user will be
told to make sure to associate the different pieces of
information so the person for whom he or she is doing the task
can easily access the information at a later date. We chose
this task because it ensures that the user use each of the
four major Palm Project applications (Address book, To-Do
List, DateBook, Memo Pad) and tests whether or not he or she
can associate the disparate information using the given
interface. For the second task, the user will be given a phone
number of an associate and asked to find all previously stored
information which is associated with this person. This task
will allow us to compare the users’ abilities to retrieve
information across interfaces. The
measures we will use will be: 1) time required for the users
to complete each task and, 2) the number of errors users made
while performing the task. Measures of time and errors will
demonstrate a comparison of learnability between the two
interfaces. If the task takes longer to perform on one
interface than another, or has more errors, this may indicate
that users had more difficulty understanding it. Time may also
indicate a comparison of efficiency between the interfaces. If
users can finish it more quickly on one redesign, it indicates
that a task can be performed more efficiently on this redesign
than another. Participants
will be given a pretest questionnaire to determine whether the
user is an experienced or novice user.
They will be asked if they have used a Palm Pilot
previously, if so, for how many years, and if they are
familiar with grafitti. The questionnaire will also ask all
users if they are comfortable with standard PC conventions
such as menus, buttons, and scrollbars, and if not, they will
not be able to participate in the study. Subjects will be
randomly assigned to the PalmProject and PalmLink treatments
to counterbalance for individual variability. We will have to
ensure that half of the experienced users and half of the new
users see PalmProject, and the other half of the experienced
and new users see PalmLink. Subjects will be paid $150 for one
hour of participation. Upon arrival into the user lab, users
will be told about the experiment procedures, and that they
will be able to quit the experiment at any time for physical
or emotional reasons. They will be asked to read each scenario
aloud before starting. The users will be informed that they
may ask questions about procedure if necessary, but not
questions of how to complete the task. Users will be asked not
to talk as that tends to slow them down, since we will be
measuring time. After signing the consent forms, each user
will be given two folders, each with a scenario describing the
aforementioned tasks. The order of the two tasks will be
counterbalanced for order effects to make sure no learning
take place from one task to the other. Thus, half of the users
will get the task of storing the hotel information first, and
the other half will first receive the task of retrieving the
associate’s information based on his phone number. The
experimenter will then give the user the Palm Pilot and leave
the room, so that he/she can observe from behind a one-way
mirror. The user will be instructed to read the first scenario
aloud, and then proceed. During the experiment, the
experimenter will record time on a stopwatch, and note errors.
The user will interact with the user as infrequently as
possible-he or she may answer questions about the logistics of
the experiment, but not about the particular task. If the user
asks for help on a task, the experimenter will not respond, or
tell the user to continue working. After completion of the
first task, he/she will be told to announce that he/she is
done. Then he/she will read the second scenario and repeat the
process. After the entire set of scenarios, the user will be
debriefed by the experimenter on the purpose of the experiment
and given an honorarium. One
of the major limitations of the experiment will be that it
will not take the biases of existing users into consideration.
Prior experience with the current system may bias the
performance of the existing users in favor of the PalmLink
system. Even though the users are instructed to use either the
PalmProject or PalmLink method of storing data, users already
conceptualize the Palm Pilot as having four or more disparate
applications with no direct connections between data.
In addition, the concept of the PalmProject was
introduced for one very specific but important Palm Pilot
function -- linking related information. Even if we find that
the project metaphor is an improvement over the current
system, we may need to do future experiments do ensure that it
comparably supports or improves other types of tasks, such as
entering events that only require the use of one or two
applications. Finally, time may be a good indicator of how
efficient a task is on one interface compared to another, but
it may or may not accurately represent difficulty. An easier
task may just take longer because it has more steps to follow
than a harder task -- it does not necessarily mean that the
user understood how to do it better. Also, if a user gives up
on a task, or runs out of time, this puts an artificial time
cap which may or may not mean the task was easier. Part 4:Thoughtful RetrospectiveThe most useful ideas for our redesign were those generated from empirical techniques such as the think aloud and contextual inquiry, rather than analytical ones such as cognitive walkthrough and heuristic evaluation. The four usability techniques ranked from most important to least important for our primary PalmProject and PalmLink redesigns are: 1) think-aloud, 2) contextual inquiry, 3) heuristic evaluation, and 4) cognitive walkthrough. There are some other redesigns that we have proposed as fixes to other major categories of problems. Most of these other redesigns are derived from Think-aloud and heuristic evaluation data, and to a lesser extent contextual inquiry and cognitive walkthrough.
We
found that the most productive technique for our purposes was
the Think- aloud.
The main advantage of the think-aloud is that it gave us a
good idea of the severity of the problems that the user
encountered. We
were able to hear the user express struggle and frustration,
even to the point of wanting to give up. Because of this, we
were able to separate the most critical incidents from
infrequent or irrelevant problems. We were able to prioritize
which problems were severe enough that they should definitely
be addressed in the design process.
The primary focus of our Palm Pilot
redesign is to allow the user easier access to and storage of
related information. Many
of the problems that we observed in the Think-aloud user study
involved the inability to efficiently connect related
information. The user was very frustrated in trying to enter
information about a meeting and having no real way to connect
the data. It was
the Think-aloud that led directly to the idea of a project
metaphor as a way to organize data on the Palm Pilot.
The
heuristic evaluation (HE) revealed numerous small usability
problems with the Palm Pilot.
Many of these problems, however, were not seen in the
Think-aloud analysis. The lack of HE problems found in the
Think-aloud, combined with the relatively low severity
rankings that most of the group members gave to almost all of
the problems, indicate that the heuristic evaluation seems to
yield many relatively unimportant problems which actual users
came across infrequently if at all.
This assertion is not denying the utility of heuristic
evaluations – it is simply that in our situation, many of
the most serious problems were found in the techniques
generated from empirical user data, rather than analytical
methods. One
of the more interesting results of the heuristic evaluation is
the different types of problems found by domain experts versus
domain novices. Many
of the problems found by those unfamiliar with the palm pilot,
such as graffiti errors and difficulty understanding how to
exit an application (the ‘non-desktop’ metaphor), were not
produced by those evaluators familiar with the Palm Pilot and
its application. This
led to several conclusions by the team: 1).
If the device being evaluated by the user is a walk-up
and use (very easy learn ability and low level of support),
then domain knowledge is not really vital.
In fact, it is probably better for the evaluators not
to have any domain knowledge. 2).
If the device has to have some level of support, such
as the Palm Pilot, many of the errors found by domain novices
are often not important or relevant. In situations where the product
requires some level of support, having some domain knowledge
is necessary for producing useful heuristic evaluations.
Otherwise, too much time is spent finding problems that
are not really encountered by actual users.
A somewhat surprising finding for our redesign was the
relevance of the contextual inquiry.
We did not realize initially that the CI would have so
many implications for our design because it did not directly
involve use of the Palm Pilot. However, many of the breakdowns
we identified earlier seemed to predict problems the user
would find in the think-aloud. For example, many of the struggles that the user had in the
CI involved the role strain caused by having too many memory
aids for all of her information. Much of the information was
related, but scattered across many different devices.
This problem with unconnected and redundant information
was quite similar to what we observed in the think-aloud.
The fact that the user had similar problems in the CI
convinced the group that modeling the work flow was as
effective a method for predicting problems as examining the
device itself, which was not the way we were accustomed to
thinking.
The
cognitive walkthrough evaluation was not as useful as either
the Think-aloud or the contextual inquiry, but it did provide
some indication of the problems the user had with entering
related data. Cognitive walkthrough seemed the most useful in
cases where there is a walk up and use interface, and you want
to determine how learnable the interface will be. This
sufficed for our type of user and tasks, but we don’t
believe it will generalize well to a variety of users and
tasks. We also found that it fixes specific problems, such as
insufficient feedback from the interface and labels that are
not clearly visible and comprehensible. However, we found the
guidelines for determining success and failure stories were
difficult to use, as well as poorly defined and therefore very
subjective in terms of interpretation. Even in our team, which
generally reached consensus quite easily, we had much
disagreement as to whether a story was a credible success or
failure. |
||||
|
|
|
|||
 |
||
| In-flight
entertainment system . Stockphone
. Allbank web . Chinese Gardens
. Architecture dept web . Real estate buying
system . eToys web
design .Car Music
System . Visage
redesign . Palm
Pilot III redesign
home | projects | hobbies | resume | links | contact
|
||
