• Collaborative Web sites offer researchers a new opportunity to study how people work together

By Meghan Holohan

As a young psychology researcher at Cornell, Robert Kraut visited bowling alleys, attended hockey games and observed pedestrians, watching them, hoping for the glimpse of a smile. After watching scores of people, Kraut and his co-authors concluded that people smile not simply to express joy, but also as a way to smooth their social relationships with others.

Their work was funded in part by a federal grant, and the paper that followed was published in the Journal of Personality and Social Psychology. It also earned a Golden Fleece Award in March 1980 from former U.S. Senator William Proxmire, for projects the senator thought were a waste of government money.

But the study had a serious purpose--to understand the evolution of human nonverbal communication. This research has been heavily cited and today, Kraut proudly cites the Golden Fleece Award on his CV, because the paper was a precursor to the then-emerging field of evolutional psychology, the science of determining why certain behaviors (like smiling) evolved.

A lot has changed since 1980, but Kraut, now the university's Herbert A. Simon Professor of Human-Computer Interaction, is still trying to understand how people communicate. Instead of heading to bowling alleys, Kraut is searching Wikipedia edit logs. And while he's not interested in whether the editors of Wikipedia pages are smiling, he does examine how they work together.

Wikipedia provides the ultimate experiment in Internet democracy. Anyone who wants to contribute to encyclopedic pages on topics ranging from military history to Britney Spears can do so. The volunteers who write these articles aren't vetted for their expertise or told what to do by a boss, and yet many articles turn out remarkably well.  

Kraut and Niki Kittur, an assistant professor of human-computer interaction, wondered how the writers organize their work to produce such high quality.  

"People think of Wikipedia as a technology, but it's actually a community," Kittur says. Measured in raw kilobytes, articles comprise less than half of the data on Wikipedia, he says, and about 40 percent consists of discussions among the participants, about dividing up work, debating various changes, and making decisions about content. Prior to graduate school, Kittur had been a consultant, so he had personal experience watching diverse groups work together.

Kraut and Kittur say that quality in Wikipedia doesn't simply come from the wisdom of the crowd--where averaging the judgments of enough people produces "truth." In fact, they're learning that having more contributors doesn't necessarily make a Wikipedia entry more accurate. Celebrity pages often have hundreds of writers, while pages about mathematical formulas have the fewest number. Yet math pages tend to be the most accurate Wikipedia pages. "A huge number of people working on a page can lead to chaos," Kraut says. "You get both expertise and energy when you have a large collection of editors working on an article, but you need to coordinate their work to get the benefits."

Big groups can lead to articles that are disorganized or not coherent, Kittur says, but large groups of contributors also excel at preventing grammatical errors and vandalism--cases where users intentionally write inaccurate information on a page. Kraut says about 35 users working on an article seems to be a "crossover" point--the best balance between accuracy and manageability.

Kraut and Kittur's studies of Wikipedia are challenging some established rules about how organizations should work. For the past 30 years, Kraut says, organizational psychologists have studied how to train and socialize new employees to ensure that they are hard-working, loyal and happy. Most people in the field believed that what is known as "institutionalized" training works best--for example, training new employees in a group and giving them common experiences before they start the job.

Perhaps this works in the real world, but Kraut and his collaborators--including Bo Choi, a Ph.D. student at CMU's Tepper School of Business and John Levine and Kira Alexander of the University of Pittsburgh's Psychology Department--noticed something different in Wikipedia. Contributors to Wikipedia projects don't get group training. Instead, they contribute most when they're personally welcomed to the project, offered assistance and given constructive criticism about the work they've done.

Although Wikipedia may just operate differently from conventional off-line groups, the more likely possibility, according to Kraut, is that the earlier researchers got bad data and drew the wrong conclusions. Because it's almost impossible for organizational psychologists to go into the workplace, those researchers relied on self-reported survey data, which is often fraught with inaccuracies. In contrast, Wikipedia allows the rare opportunity to watch how people work. Since all communication between "Wikipedians" takes place online, and all of their messages and edits are logged, Kraut and Kittur have a complete record of all interactions that took place.

In addition to studying live Wikipedia pages, Kittur and Kraut also have designed controlled studies to see how people work in democratic groups, setting up multiple separate "wikis." Experiment participants who are now being recruited will collaborate on a variety of simple tasks designed by Kraut and Kittur. The investigators hope the carefully measured studies will provide more answers about how people work together online--and perhaps in the real world.