From ScribbleWiki: Analysis of Social Media

Introductions and requests: Rhetorical strategies that elicit response in online communities

Authors: Moira Burke, Elisabeth Joyce, Tackjin Kim, Vivek Anand, and Robert Kraut

Paper: Introductions and requests: Rhetorical strategies that elicit response in online communities (pdf)

This paper presents a multi-method series of studies into community responsiveness to two rhetorical strategies: self-disclosing introductions and making requests. It expands the cross-sectional study of newsgroups presented in Arguello et al, CHI 2006 from approximately 6,000 messages from 8 groups to approximately 40,000 messages from 99 groups, and follows with a pair of experiments in which introductions and requests are added or removed from previously posted messages and reposted to Usenet groups. Experimental results show that introductions referencing the group history cause an increase in reply counts, but requests do not.

Experiment 1: Correlational analysis of 41,000 Usenet messages

Method

The sample was drawn from the metadata and full message text of 99 Usenet newsgroups from June 2003 to February 2005. The 99 groups represented a wide variety of topics and popula-tions, comprising four general categories: health support, technical, hobby, and political issue discussion. Health support topics include asthma, epilepsy, breast cancer, and food allergies. Technical groups include C program-ming, civil engineering, and Windows NT security. Hobby groups include quilting, the Grateful Dead, and vegetarian cooking. Issue groups include gun rights, economics, and agnosticism.

A sample of 40,931 messages was selected from the 2,179,999 messages posted to the 99 groups during the focal period. The messages were selected to be first posts in their threads, and thus were potential conversation starters rather than replies to ongoing conversations. A maximum of 500 of these potential thread-starters was randomly selected from each of the 99 groups, such that the authors were distinct and the full message text was available. Slightly more than half of the messages (57%) received a reply.

The dependent measure was a binary value indicating whether a particular message got a reply.

Features

All of the features from Arguello et al, CHI 2006 were included as controls, and the "testimonial" and "question" models were slightly improved (both were modeled using Boosted Stump Learner in Minorthird, with Kappa values of .61 to .63).

Results

Overall, messages had a 57% chance of getting a reply and the rhetorical content had a strong impact on whether the community responded. Posts that included introductions detected by Minorthird were 7% more likely to receive a reply than those without. Messages with requests detected by Minorthird were 6% more likely to receive a reply than those without.

Experiments 2 and 3: Introduction and Request Manipulations

Experiments 2 and 3 looked at the impact of two kinds of introductions:

• group introductions that describe past lurking or participation in the group (e.g. "I've been lurking here" or "I have read here before that . . .")
• topic introductions that describe personal experience with the topic the group cares about (e.g. "I was diagnosed with lymphoma two years ago" to a cancer group, or "I'm running Apache 2 on Red Hat 9" to a Linux group)

Method

Of the 99 groups from Study 1, 93 were still active in July 2006. From each of the 93 groups, two potential thread-starting messages—not replies to existing threads—that were at least one year old were randomly selected from the Google Groups site. Spam, FAQ reminders, outdated topics, and obvious flaming were excluded. Messages were selected such that one had an explicit question and one did not.

In Experiment 2, three versions of each message were created such that each base message had a version without an introduction, a version with a group intro, and a version with a topic intro.

Examples of versions of one base message:

In Experiment 3, six versions of each message were created, combining explicit questions (or the lack of an explicit question) with each level of introduction.

The outcome was a count of the number of replies to each message, controlling for the number of replies it received when originally posted, and the type of group it was posted to.

Results

In Experiment 2, adding a group introduction roughly doubled the number of replies, from an average of .42 to .81. Adding topic introductions did not significantly affect the number of replies. The authors hypothesize that this could be due to the relative genericness of their topic introductions. The presence of an explicit question was correlated with an increase in replies by 68% to 0.71, though causality of explicit questions was not tested in Experiment 2 (it was in Experiment 3). The results of Experiment 3 were less clear; the authors hypothesize that the introduction of both question and introduction manipulations watered down the effect of introduction manipulations, and so they performed their analysis on the subset of the messages in which only introductions were manipulated, and found that group introductions did increase reply rates again. Future experiments are proposed, in which different kinds of requests (e.g. implicit, explicit, or none) are manipulated.