A Feature-based Comparison of Software Preferences in
Typically-developing Children versus Children with Autism Spectrum Disorders

Jill Fain Lehman, Ph. D.
Senior Research Computer Scientist
Carnegie Mellon Univerisity

Copyright 1998, Jill Fain Lehman. All rights reserved.

Introduction

In 1996 I became involved with parents and professionals who work with children with autism spectrum disorders (ASD), providing suggestions about commercially-available early education software that might be considered therapeutically or educationally relevant for this population. Like many young typically-developing children, many children with ASD find computers engaging. Indeed, reports by both parents and professionals often indicate a preference for computer activity over other sorts of play and the kind of intense and sustained attention during interaction that is a prerequisite for learning. While children with ASD may find commercial software engaging, however, they may do so for reasons that are different from those of typically-developing children. These questions,

What features of early learning software do children with ASD find engaging or motivating? and
Are they the same features that engage typically-developing peers?

are important not just for the evaluation of existing software, but for the design of software specific to the special needs of this population as well.1

To examine these two questions it makes sense to begin with a list of features that we know are important to young, typically-developing (TD) software users and then ask whether such a list also reflects the preferences of users with ASD. The beginnings of such a list can be found in early empirical work by Malone on the design of motivating instructional environments.2 Surveying 65 children ranging from kindergarten through eighth grade, he had them rate the subset of 25 computer games with which they were familiar on a three point scale (don't like, like, like a lot). Each game was then analyzed for the presence or absence of 10 features, based on the characteristics of computer games at the time and theories of motivation by Banet 3 and others. The features Malone used were:

  1. Existence of an explicit goal
  2. Computer keeps score
  3. Audio effects
  4. Randomness involved in game
  5. Speed of answer counts
  6. Visual effects
  7. Competition
  8. Variable difficulty level
  9. Cooperation
  10. Fantasy
Malone found high positive correlations between user preference and the presence of the first five features (p<.01 for goal, score, audio and randomness, p<.05 for speed). He went on to show that some of these relationships were actually causal, manipulating preference by systematically varying some of the features in different versions of two popular games.

There are three advantages to basing our own survey on an updated version of Malone's original list. First, of course, is the existence of prior results that show high positive correlation in typically-developing children. The second advantage is that the list itself is based on the application of a number of cognitive and educational theories of intrinsic and extrinsic motivation to the problem of learning in general and software design in particular.

The third and most compelling advantage of starting with Malone's list, however, is that most of the features on it constitute areas of experience in which we might a priori expect to find differences between typically-developing children and those with autism. Such expectations would be based on differences between TD and ASD children that are part of the defining characteristics of the syndrome. For example, we might expect that visual effects would be more important to children with ASD since they tend to rely on visual reasoning and visual organization of information. Similarly, we might expect the existence of randomness and variable difficulty levels to be negatively correlated with use for children with ASD who generally seek predictability in their environments. Speed might also be a negative factor for those children whose symptoms include motor planning impairments. Further, it is assumed that most children with ASD do not have the kind of ability to reason about an opponent's strategy that makes competition challenging and fun, that they prefer solitary to cooperative play, and that they are incapable of or are highly restricted in their fantasy. These characteristics of the syndrome, then, predict group differences for the features competition, cooperation, and fantasy. Expectations with respect to explicit goals, scoring, and audio effects are more equivocal. It is possible to imagine that conventional goals and scoring imply competition and would, therefore, be unimportant; but it is also possible to argue that by representing the goal and scoring visually such features would become both salient and motivating. Audio effects such as music might be an important enhancement whereas some sound effects might be particularly aversive for those children with auditory hypersensitivities.

Of course, a plausible story for why there might be group differences does not guarantee that such differences will outweigh individual variability. However, the fact that we have such a priori expectations means that by comparing the preference for these features in software activities with the preference for the same features in non-software activities, we may be able to find evidence pertaining to a third question:

Are children with ASD more like their typically-developing peers when they are involved in computer interactions rather than non-computer interactions?

Such a question has important implications for the special education community, as it begins to address the issue of what role computers might have in levelling the educational playing field.

We have stated the three questions central to our survey's design as questions rather than hypotheses to underscore the exploratory nature of this work. Because the survey was exploratory, we offered a large number of features as potential discriminators between the two groups of children, including but going beyond Malone's original list. Although this has implications for achieving statistical significance given a moderate group size (as discussed below), it seemed the appropriate course of action for a preliminary study.

Description of the Survey

The survey consisted of a brief introductory explanation of the motivation behind the study followed by an instruction section and six multi-part questions. The first three questions collected general information about the child, including age, sex, diagnosis of a disability, relationship of respondent to child, and information about educational placement and communicative and literacy skills (the lack of socio-economic data is discussed below). Since there is a great deal of variability in the assignment of particular diagnostic categories within the autism spectrum (e.g., high-functioning autism versus pervasive developmental disorder versus Asperger's Syndrome), educational placement and communicative-level (especially when compared with chronological age) seemed likely to provide a better indication of level of functioning than diagnosis alone.

Question 4 collected information about the child as a computer user, including who was responsible for purchasing software, what criteria were used, age at which independent interaction was mastered, use of an accompanying manual for the software, and choice of hardware platform. This question was intended as a gross measure of the comparability of the two populations as software users. Significant differences along these dimensions could introduce a potential bias in the result of the feature analysis. For example, if the TD children chose their own software primarily for entertainment purposes and the parents of the ASD children chose primarily therapeutically-oriented software, then the a priori distribution of some features within the software could be different, and thus, the perceived relative importance of the features could be different as well. Question 5 was intended as a refinement to this gross measure and asked the respondent to list the child's three most and three least favorite software titles.

Primary Measures of Interest

Information relating to the three questions presented in the introduction was gathered using survey questions 6a and 6c. These two questions presented lists of software features and asked for ratings of each feature using the following scale:

1 = Dislikes it, 3 = Doesn't care, 5 = Very important

The scale is unusually worded because some children with ASD have a limited emotional range or limited ability to show emotion, especially positive emotion, in conventional ways. A scale with endpoints labelled Strongly dislikes/Strongly likes thus seemed inappropriate. A scale with the endpoints Absence very important/Presence very important seemed problematic as well; how do you judge whether the absence of something is important? The rather unbalanced verbiage shown above was an attempt to try to solve this problem. However, a scale labelled Avoids activities with this feature/Prefers activities with this feature might have been a better, and more conventional, choice 4.

The main difference between questions 6a and 6c was the set of activities to which the features were to be related. Question 6a asked participants to rate the following 23 features in the context of the child's computer activities:

Question 6c presented a subset of 20 features from this list, but asked for ratings in the context of non-computer activities. The list in 6c omitted sound effects 5, use of animated character guides, and use of on-line help, and generalized some of the other descriptions (e.g., A variety of activities on the disk became simply A variety of activities). Question 6b asked participants to list the particular themes or subject matter of interest if the presence of such topics was an important software feature for the child.

The list of 23 features given above both subsumes and extends Malone's. In five instances the description of one of Malone's features was changed. Existence of an explicit goal and Computer keeps score were changed to Clear goal and Visible reward in an effort to more clearly distinguish the existence of the goal per se from the use of a visible cue to signal progress toward the goal. Variable difficulty level was changed to Multiple difficulty levels to better match the language used in marketing software. Cooperation was changed to Ability to use with another person to contrast this feature more clearly with the added feature Ability to use alone (one may argue that this change alters Malone's intended meaning but it is, for our purposes, the more pertinent choice). Finally, Fantasy was generalized to Involves fantasy or role-playing.

Features were added for one of three reasons. In two cases, we simply teased apart one feature of Malone's into multiple related features. Malone's list includes Audio effects whereas ours distinguishes Sound effects from Music. Similarly Visual effects is split into Visual effects such as animation versus Rich visual detail. In three cases we added features Malone would not have seen in 1980 but that seem characteristic of early learning software today (Use of animated character guides, On-line help, Variety of activities).

Although Malone's list contains many features that we might expect to be differentially preferred by children with ASD, it was never Malone's intention to derive his list with an eye toward the special needs of this population. Thus, in the remaining eight cases additional features were added because they reflect important characteristics of current software specifically with respect to the information processing needs of children with autism. The features No reading required and Text in addition to auditory information were included because many children with ASD have difficulty processing auditory information. Software without text generally relies more heavily on auditory presentation of information, whereas early learning software that includes text tends to present information redundantly to the visual and auditory channels. The features Relevant theme and Particular subject matter were added in response to the restricted interests and fixation on specific topics that are one of the diagnostically-relevant characteristics of the syndrome. The inclusion of a feature to detect a preference for Fact-based activity was motivated by the same literature that argues for a limited ability in fantasy and pretend play. The remaining features--- Ability to use alone, and Open-ended/child-controlled activity versus Program-guided activity ---were motivated by the defining characteristic of autism, a preference for solitary play and internal locus of control.

Data Collection Method: Subjects and Procedure

The survey described above was distributed and collected via the Internet. The decision to use the Internet was based on three factors: cost, speed, and identification of the target disabled population. Organizations like the Autism Society of America do not make their membership lists available out of respect for members' privacy. Advertising in relevant publications is costly and space requirements for such advertisements preclude including the questionnaire itself. This, in turn, requires multiple responses by participants (first to identify themselves from the advertisement, then to send in questionnaires). Each additional response is likely to decrease the rate of return. In contrast, using established Internet list servers (or listservs) allowed us to reach potential participants quickly, at virtually no cost and with minimal intrusion into their time or privacy. Still, it should be noted, that by performing an Internet-based survey the results may be significantly biased; individuals with access to the Internet who are likely to subscribe to listservs are a small, self-selected subgroup of the general population. Although they are, perhaps, representative of the subgroup who are likely to have children who use educational software on a regular basis, the bias in the sample should always be kept in mind when interpreting or discussing the data.

Survey distribution was performed in two stages for each of the two populations, the ASD group and the TD group. In the first stage, an invitation to participate in the survey was posted to appropriate listservs. Listserv readers who responded to the post were then sent surveys via email. Surveys were, for the most part, completed on-line and returned via email. In a few instances, differences in email programs made this impossible and surveys were returned in hard-copy form via the postal service.

The invitation for the ASD group was posted to the lists autism@maelstrom.stjohns.edu and aspberger@maelstrom.stjohns.edu at the end of April, 1997. The request for participation was later forwarded by one respondent to a group of parents interested in hyperlexia (a characteristic that is sometimes found among high-functioning children with ASD). The post invited response from individuals willing to fill out a short electronic questionnaire who had a child (or children) who used commercial, off-the-shelf software in a home, therapy, and/or school setting and who had been diagnosed with any of the autism spectrum disorders (autism, pervasive developmental disorder, Asperger's syndrome, etc.).

Although it was easy to identify a likely source of potential participants for the ASD group, finding a comparable on-line source for the control group of typically-developing children was more difficult. Since participants from the autism community were self-selecting on the basis of an interest in children's use of educational software, we targeted lists for the TD group that appealed specifically to individuals interested in this topic as well: chi-kids@acm.org, netnews.misc.kids.computer, netnews.alt.comp.shareware.for-kids, netnews.alt.education.home-school, netnews.misc.kids, and crc@classroom.net. Most of the responses we received for the ASD group in the first week were for pre-teen children who were using software in the home. Thus, to maximize the percentage of usable data from the TD group, the post to the TD lists was revised slightly from the original to request individuals willing to fill out a short electronic questionnaire who had a pre-teen child (or children) who used commercial, off-the-shelf software in the home.

In collecting the data, no socio-economic information was requested of participants. Socio-economic equivalence of the two groups seemed likely by virtue of choosing Internet listserv readers who had personal computers in the home. While it is a weakness of this exploratory study that such an equivalence was not established empirically, we had to weigh the usefulness of such a comparison against the possibility of reduced participation due to concerns about providing this kind of information, especially over the Internet in electronic form.

Results

Response Rate

We received 88 requests for questionnaires from individuals associated with children in the ASD group. Of those, 58 individuals (66%) returned one or more surveys; two individuals returned surveys for two children. Of the base number of 60 surveys, four were excluded from further analysis on the basis of the age of the child (older than twelve), two were excluded on the basis of inadequate information (defined as rating fewer than 2/3 of the features listed in question 6a or question 6c), and one was excluded on the basis of diagnosis (Landau-Kleffner Syndrome, which shares autistic symptoms but is not defined as an autism spectrum disorder). After exclusions, there remained 53 responses for the ASD group. All responses for this group were provided by the child's parent or guardian (see question 1d).

We received 77 requests for questionnaires from individuals associated with children in the TD group. Of those, 45 individuals (58%) returned one or more surveys: three individuals returned surveys for two children, and two individuals returned surveys for three children. Of the base number of 52 surveys, five were excluded on the basis of inadequate information (two individual children and one of the sets of three children from a single household) and three were excluded on the basis of disability (one case of aphasia and two cases of speech problems that required some support within a mainstream classroom environment), leaving 44 responses for the TD group. Of these 44 responses, 40(91%) were provided by the child's parent or guardian, three were provided by a family member other than the parent, and one was provided by the child's teacher.

Population Characteristics

Survey questions 1 through 4 served as the basis for characterizing the ASD and TD groups. In the following table we compare the populations in general, below we focus on their characteristics as computer users:

General Population Characteristics
* denotes significant difference, p<.01
** denotes marginal difference, p=.09
ASD TD
Number of children 53 44
Mean age 6.4 years 6.0 years
**Male(Female) 81%(19%) 66%(34%)
*Disability No disability 0% 100%
Higher Functioning 49% 0%
Lower Functioning 51% 0%
Readers 72% 60%

Disability is the only significant difference between the groups for these characteristics. Overall the ASD group may have come from the less-impaired portion of the spectrum: only 9% (5 children) were non-verbal, compared with reported rates of 50-75% in the literature. The distinction made here between higher and lower functioning children with ASD was predicated on a combination of educational situation and communication level. A lower-functioning child was defined to be one who required a special education classroom or whose communicative level was well below norm for chronological age (including, for example, a five year old child who was verbal but reported to be at the two- to three-word stage). All children considered to be higher-functioning were home-schooled, partially or fully mainstreamed and communicated verbally in full sentences.

The difference in male/female ratio between the groups reached only marginal significance. Since autism differentially affects males at a ratio of about 3.5:1, in any sample of this population we would expect the distribution of males and females to be similar to the one found here. All other things being equal, we would expect the split for the typically-developing group to be 50/50, resulting in a significant difference between groups for this characteristic. The male bias in our data for the TD group may reflect gender-related preferences in young children for computer activities, or may have some other cause. However, the disparity between the groups on this measure seems inevitable given the skew in the male/female distribution for the ASD population. We address the potential implications of this disparity below.

As shown, there was no significant difference in the number of readers in the two groups (Chi-square = 1.35). The distribution of readers among the higher and lower-functioning subgroups was fairly even, with 16/38 (42%) readers categorized as lower-functioning and 22/38 (58%) as higher-functioning.

In addition to being generally comparable, the two groups appear to be drawn from essentially the same population of computer users, at least for the gross measures we considered:

Computer-related Characteristics
** denotes marginal difference, p=.10
ASD TD
Mean age at mastery 4.3 years 3.8 years
**Macintosh users 26% 43%
Software purchaser Adult only 55% 52%
Adult & child 45% 41%
Child only 0% 7%
Adult purchase criteria Education first 76% 80%
Other 24% 20%
Manual readers 23% 20%

On average, children in both groups mastered basic interaction via mouse or keyboard around their fourth birthday. The youngest age of mastery in both groups was 1.5 years, and the standard deviation for mastery in both groups was 1.8 years.

As noted in the table, the difference between groups in the use of Macintosh versus PC platform is marginally significant, a fact for which we have no obvious (or plausible) explanation. To understand whether the difference might have had an impact on which software the children used (and, thus, on which features they disliked or found important), we can compare the software titles given in response to question 5 of the surveys. Analysis of the data from question 5 shows usage in both groups is dominated by popular titles that are available on both PC and Macintosh machines.

In almost all cases the adult responding to the questionnaire was solely or partially responsible for the purchase of software. Overwhelmingly, the primary criterion for purchase was educational. Only 20% of the TD group chose entertainment as the main reason for choosing a software title. For the ASD group, 11% gave entertainment as the main reason and 13% chose software primarily for therapeutic use.

Feature Analysis

Features pertaining to software interaction split into two groups: 3 features were defined in the computer context only (Sound effects, Animal characters, and On-line help), while the remaining 20 features were defined in both the computer and non-computer contexts. Means for all features are given next, with the 3 computer-only features listed in the first three rows.

Means Table for Features
ASD Group TD Group
Computer Non-Computer Computer Non-Computer
Sound effects 4.44 --- 4.39 ---
Animal characters 3.98 --- 3.60 ---
On-line help 2.98 --- 3.05 ---
Goal 3.84 3.98 3.86 3.89
Visual reward 4.20 4.13 4.02 3.79
Music 4.34 4.09 4.20 3.91
Randomness 2.88 2.27 3.48 3.16
Speed 4.74 3.19 3.65 3.60
Visual effects 4.74 4.55 4.65 4.05
Competition 2.55 2.49 3.25 3.57
Multiple levels 3.53 3.04 3.73 3.74
Fantasy 3.09 3.21 3.58 3.79
No reading 3.23 3.17 3.68 3.56
Text and audio 3.65 3.73 3.12 3.21
Visual detail 4.71 4.58 4.18 4.07
Theme 4.06 3.96 3.91 3.74
Subject 3.96 4.09 3.70 3.82
Child-controlled 4.04 4.43 4.15 4.30
Program-guided 3.40 2.65 3.19 3.00
Factual 3.02 3.25 3.15 3.30
Variety of activities 4.11 3.87 4.25 4.36
Use alone 4.55 4.42 4.28 4.00
Use with others 2.94 3.05 3.80 4.09

Recall that the large number of features to be considered in looking for differences between the ASD and TD groups is a function of the exploratory nature of this survey. Unfortunately, a side effect of the feature set size is the relative reduction in power of the statistical tests for the given population sizes and, therefore, the corresponding adjustment of the significance level to p <= .002 6. However, since the study is exploratory, we will want to consider in our discussion values that might well have reached significance in a survey with a more focussed feature set. Consequently, we include in our reporting values for which p < .01 as well.

The difference between diagnostic groups for the three computer-only features (Sound effects, Animal characters, and On-line help) were evaluated by t-tests with none reaching significance (p = .62, p = .03, and p = .67, respectively). The remaining 20 features were evaluated using a repeated measures analyses of variance with diagnosis as a between-subject factor and environment (computer vs. non-computer) as a within-subject factor. The next table shows those features having significant differences between and/or within groups. None of the interaction terms (diagnosis x context) was significant.

Repeated Measures ANOVA for Software Features
* denotes significant difference, p<=.002,
** denotes a feature of interest, p<.01
ASD vs TD Computer
vs Non-computer
Randomness F=16.770
p=.000*
F=11.550
p=.001*
Visual effects F=8.615
p=.004**
F=17.768
p=.000*
Competition F=14.409
p=.000*
---
Multiple levels F=8.200
p=.005**
---
Text and audio F=11.565
p=.001*
---
Visual detail F=14.500
p=.000*
---
Use with others F=25.921
p=.000*
---
Music --- F=10.279
p=.002*
Child-controlled --- F=7.103
p=.009**
Program-guided --- F=9.771
p=.002*

The column labelled ASD vs TD examines significant differences between the groups in their attitude toward features independent of whether those features are found in a computer or non-computer environment. In other words, preference for each of Randomness, Competition, Text and audio, Visual detail, and Use with others was significantly different as a function of disability with the computer environment neither contributing to nor detracting from the effect. Examination of the means, collapsed across contexts, shows that Randomness, Competition and Use with others are differentially preferred by typically-developing children, while Text and audio and Visual detail are differentially preferred by children with ASD. Further, Visual effects is a feature of interest that children with ASD may prefer more than their typically-developing counterparts. Similarly, Multiple levels of difficulty may appeal more to typically-developing children in all contexts.

The column labelled Computer vs Non-Computer examines significant differences between contexts independent of disability. In this case, means reveal preference for each of Randomness, Visual effects, Music , and Child-controlled activities were all greater in the computer context, regardless of whether the child was autistic. Interestingly, Program-guided activities also showed a marginal context effect independent of disability.

Discussion

In the introduction we suggested that Malone's original list of features was an appropriate starting point for this study because most of the features reflected areas in which we might expect to find differences between typically-developing children and children with autism. Then, in extending Malone's list, we did so, for the most part, with an eye toward the particular information processing needs of children with ASD. In all, it seemed that we could make reasonable a priori arguments for differences between the diagnostic groups for sixteen features, while the remaining seven features, if a source of differences, would require post hoc analysis. Yet, out of 23 possible points of comparison only 10 features showed statistical or near-statistical differences, and only 7 features showed differences based on disability. In the remainder of this section we discuss the results for each software feature in the context of our expectations.

Expected and Found

Instances where we expected a difference based on disability and found one include the features Randomness, Competition, Text and audio, Visual detail, Use with others, and (marginally) Multiple levels of difficulty, and Visual effects such as animation. The existence of text along with auditory information and the inclusion of rich visual detail are more important to children with ASD than to typically-developing children regardless of context. In other words, using a software environment for education does not in and of itself compensate for this differential need in children with ASD. Indeed, it may be that a strong component of the preference for computer interaction in this population is a response to the medium's essentially visual approach (especially in contrast to the typical classroom's reliance on auditory instruction). Nevertheless, an attraction to computers and preference for computer interaction does not level the playing field; reliance on visual presentation of information remains a key feature for engagement even on computers and is, thus, a critical component of software for these children.

In contrast, Randomness, Competition, and Use with others are differentially preferred by typically-developing children (recall that randomness was one of the critical features for engagement found in Malone's original study). As a negative factor for children with ASD, Randomness and Competition may be particularly problematic because they feature prominently in current software design. Within-group measures show that children with ASD are significantly more willing to accept randomness in a computer context than in a non-computer activity; however, in either context the average attitude toward randomness is still on the negative side of the scale.

Competition shows no such ameliorating effect by context and is similarly negatively viewed by children with ASD. Further, the case for a between-groups difference for Competition may be even stronger than our data suggests. Given the amount of data collected, it was inappropriate to try to include gender as a potentional factor in our analysis. The literature on gender differences in computer use, however, indicates that a competitive feature in software is differentially preferred by males. Our data shows that the ASD group disliked competition despite being predominantly male (81%) while the more balanced TD group (66% male) rated competition favorably overall (that is, despite a higher percentage of females than the ASD group had). Thus, if we consider the scores for Competition in light of the gender ratios of our two groups, it seems likely that the difference in preference, at least among males, is even larger.

Use with others, showed a small positive preference by the TD group but was viewed as essentially unimportant by the ASD group (the mean value collapsed across contexts corresponds to the doesn't care choice on our scale). This is the least problematic of the discriminating features, however, since most software that can be used with others can also be used effectively alone. Moreover, using the same logic about gender ratios applied above, it may be the case that the difference detected for Use with others is artificially inflated. If, as the literature suggests, it is the female portion of the TD group that contributes the positive scores for this feature, then the imbalance in gender percentages between the groups may be the source of the effect. To see if Use with others (and Competition) show gender-related differences for the ASD group in this age range would require collecting significantly more data.

The seventh feature to show an effect for diagnosis was Multiple levels of difficulty. The essence of the difference seems to be a relative lack of importance for the ASD group compared to a definate preference for the TD group. We conjectured in our initial discussion that the characteristically rigid play of children with ASD might translate into a tendency to try to use software in the same way time after time. However, it also seems possible that children with ASD simply do not notice the change in difficulty level or do not derive satisfaction from attaining higher levels. An additional survey question needs to be designed to tease apart these possibilities.

Expected but Not Found

In addition to Text and audio, seven other features were added to Malone's list in response to differences we see between children with ASD and typically-developing children in non-computer contexts. None of these features reached significance in the between-groups comparison.

Since the deficits of children with ASD may include impairment in motor planning, we thought that programs requiring Speed of response might show a negative preference effect. In fact, the ASD group showed a higher mean preference for speed in the computer context, although this interaction term (diagnosis x context) did not reach significance. As noted above, it is possible that our ASD group represents the higher-functioning portion of the spectrum, that is, children with fewer impairments overall. It is also possible that the predominant use of educational software---which tends to have response requirements considerably more in line with average performance than do video games---contributes to the explanation.

Inclusion of the features Fantasy or role-playing, Relevant theme, Particular subject matter, and Fact-based activity was predicated on the restricted imaginative play of children with ASD. Again, it may be that the lack of effect for these features was due to a higher-functioning sample than is representative of the autistic population as a whole. Alternatively, it may be that even typically-developing children in this age range prefer software that responds to the desire to explore the particular theme or subject-matter of current interest. Whether this says something inherent about the children or about the quality of this type of software (as opposed to either fantasy or fact-based programs) cannot be discerned by our survey.

The feature No reading required was a counterpart to Text and audio and was expected to be more preferred by typically-developing children who can make use of the auditory information that is a major component of early learning software. Means were higher for the TD group (and highest in the computer context), but none of the differences reached significance and may have simply been the result of the ASD group containing a higher percentage of readers.

Although Ability to use with others showed an effect for disability, strictly speaking Ability to use alone did not (p=.02). Certainly the trend in the data is what we expected, with means uniformly higher for the ASD group. The data suggest to us that typically-developing children may use the computer to fulfill much of their need for solitary activity, but that that need is less strong than in their peers with ASD.

The important distinction in early education software between discovery environments and drill-and-practice was reflected in the features Child-controlled activity versus Program-guided activity. We conjectured that the autistic child's preference for solitary play and control over the environment would be reflected in higher scores for Child-controlled activity and lower scores for Program-guided activity compared to their typically-developing counterparts. Although they showed no effect for diagnosis, both features showed a marginal differential effect for context. Means indicate that child-centered activity is preferred to non-child-centered activities by all the children regardless of context, but having control in non-computer contexts was more important to children in both groups. To put it slightly differently, all the children seemed more willing to abdicate some control in the computer context.

Since the surveys were not being answered directly by the children, it is particularly important that a number of features that were expected to show differences did not do so. Most parents who have a child diagnosed with a disability like ASD will be familiar with the sorts of behavioral differences expected in the population. We cannot ignore the possibility that a parent's response would be based on what is generally true about children with autism, rather than what is specifically true about his or her child. We consider the lack of effect in features such as Fantasy and Theme, and the unexpected reversal of preference in Speed to be good indicators that parents were not simply giving rule-of-thumb responses.

Not Expected and Not Found

The remaining seven features included four based on Malone's original list---Clear goal, Visible reward, Sound effects, and Music (the latter two deriving from the original Audio effects)---and three that were added because they are prominent in early learning software today---Use of animated characters, On-line help, and Variety of activities. No significant differences for these features were expected a priori based on diagnosis, and no between-group differences were found.

Recall that Clear goal, Visible reward, and Audio effects were three of the four features that showed high positive correlations to user preference in Malone's study (the other was Randomness, a feature that showed significant group differences in our data). Sound effects and Music also had some of the highest mean scores in both of our groups, with Music showing a significantly greater importance in the computer context for all the children. Variety of activities also scored high in the computer context. The means for Clear goal, the most highly correlated feature in Malone's study, were almost indistinguishable across the grouops in either context. Clearly, the similarity of the children along these dimensions helps explain the enjoyment of off-the-shelf software by children with ASD, despite their many differences with respect to typically-developing peers.

Although we made a number of a priori conjectures about which features might show group differences, our third question---Are children with ASD more like their TD peers when on a computer?---was strictly exploratory7. The interaction terms of the repeated-measures ANOVA (diagnosis x context) address this issue. Although none of the interaction terms reached significance, two seem to be worth further attention: Visual effects (p=.03) and Competition (p=.04). In the case of Visual effects there seems to be the potential for an interaction such that children with ASD can be said to be even more sensitive to visuals on the computer than they are to visuals in non-computer activity when compared to typically-developing children. Given the independent differences both between-groups and within-group and the high means in all conditions, it seems possible that the interaction was obscured by a ceiling effect. With respect to Competition, the potential direction of the interaction corresponds to an even higher aversion to competition on computers. As described above, such an interaction may have been obscured by gender percentages in the two groups.

Conclusions

In May of 1997 we conducted a pilot Internet survey comparing the use of educational software by children with autism spectrum disorders to its use by typically-developing children. The questions asked were intended primarily to help us understand which features of software are important and engaging to children with ASD and whether those features differ from the features that are engaging to children who are not developmentally delayed.

We found significant differences between the groups with respect to five common features of software. The use of randomness in presenting information, the use of competition as a motivating factor, and the ability to use a program cooperatively with others were all more important to typically-developing children, while the complementary use of both text and audio information and richness of visual detail were differentially important to children with ASD. The data was suggestive of two additional effects: preference for visual effects such as animation in the ASD group and preference for multiple levels of difficulty in an activity in the TD group.

Contrary to expectations, the children with ASD did not have a stronger preference for thematic, subject-restricted or factual material, nor were they significantly more averse to fantasy or role-playing activities. Most surprising was a strong attraction in the ASD group to educational software that involved speeded response.

The similarities between the groups were just as interesting as the differences. Both sets of children responded strongly to the use of music and sound effects in their software and preferred child-controlled to program-controlled activity. The children in this age group seemed to feel somewhat positively toward the use of animated character guides and neutrally with regard to on-line help facilities. Although the TD group had a significantly higher preference for using software with others, both sets of children enjoyed the computer primarily as a solitary pursuit. Scores also showed no distinction between the groups with respect to the importance of a goal or visual reward in computer play.

The role of computers in our educational system is of increasing economic and philosophical concern. Real inequities in access to the technology have been recognized for many years and many populations, including children with developmental delays. But the question of physical access, while fundamental, may be ultimately less important than the question of intellectual access. A computer in the classroom is only as useful as the appropriateness of the software it runs. The pilot study reported here is a preliminary step in understanding how the information needs of one population may differ from the norm. In addition to being an interesting comparison in its own right, two purposes are served by studying this question. With such information:

  1. it may be possible to articulate a set of rational design principles for educational software for this population, at least with respect to the differences revealed. As an example, the information from this study has already been incorporated into the design of a software environment for language intervention for children with ASD.
  2. it may be possible to create a metric for evaluating off-the-shelf educational software with the special needs of children with ASD in mind. Such a metric could be used both as an organizational tool for a database of commercial software and as an objective set of criteria for recommending such software to therapists, educators, and parents.

A larger-scale survey, focussing more narrowly on the features suggested by our preliminary data, would undoubtedly contribute to these goals.

Endnotes

1 Indeed, my own research involves building an interactive environment for language remediation for verbal, language-delayed children. One of our goals in designing the system has been to use the engaging characteristics of commercial early learning software in a clinically-informed way. back

2 Malone, Thomas W., Toward a Theory of Intrinsically Motivating Instruction, Cognitive Science, Volume 4, 1981. Note that while there are many current published ratings of early learning programs (and some of these are based on appraisal by young users), none includes a feature-based analysis that correlates presence or absence of a set of features with eventual approval or disapproval. back

3 Banet, B. Computers and early learning: A new direction for High/Scope Foundation. Calculators/Computers, Volume 3, 1979. back

4 Of the 103 respondents to the questionnaire, only one commented on the unusual nature of the rating scale. In general, participants did not seem to have trouble interpreting the scale, using the full scale and occasionally introducing non-integer values (e.g., 3.5). back

5 In retrospect, sound effects probably should not have been omitted, being an increasing component of non-computer electronic games). back

6 The adjusted p-value is found by dividing the usual significance level (.05) by the total number of tests to be performed on the data (23). back

7 For the sixteen features we included in the list because of clinical differences between children with ASD and their TD counterparts, it might reasonably be expected that we would have found significant between-group differences in the non-computer condition. However, by using the repeated-measures ANOVA, significance in the diagnosis x non-computer condition is defined relative to differences in the diagnosis x computer condition (i.e. the two tests are not computed independently). Without significantly more data (or a reduced feature set) we can only look to the means for justification of our initial reasoning. The means table shows that in all instances but one (Factual material) the difference in preferences between the diagnostic groups for the feature in the non-computer condition is in the expected direction. For example, Fantasy shows means of 3.21 (ASD) versus 3.79 (TD) while Subject shows means of 4.09 (ASD) versus 3.82.back