From ScribbleWiki: Analysis of Social Media

Talk to Me: Foundations for Successful Individual-Group Interactions in Online Communities

Authors: Jaime Arguello, Brian Butler, Elisabeth Joyce, Robert Kraut, Kimberly S. Ling, and Xiaoqing Wang

Paper: Talk to Me: Foundations for Successful Individual-Group Interactions in Online Communities (pdf)

In an observational study of approximately 6000 messages from 8 Usenet newsgroups, the authors identified factors related to community responsiveness and member retention. They found newsgroup, individual, and linguistic-level factors that predicted whether a particular post would get a reply, and whether a poster would post again to the group.

Method

The corpus included metadata and full text of 6,174 randomly sampled messages from eight Usenet newsgroups on health support, political, technical, and hobby topics. Only the first post in a thread was selected; the authors were interested in analyzing attempts to start conversations, rather than replies to ongoing conversations. Quoted text and headers were removed from the messages, which were then stemmed and parsed and run through the LIWC (Linguistic Inquiry and Word Count) application to count words appearing in psychosocial dictionaries, such as emotion words.

Two rhetorical strategies were classified using SVM in Minorthird: personal introductions ("testimonials") and questions. Introductions like "I was diagnosed with leukemia last month" or "I've been reading here for a few weeks" were coded by a human judge on a set of 200 training messages. Ten-fold cross validation had a recall of 0.89, precision of 0.89, and Kappa of 0.78. Questions, including indirect requests like "I'm looking for . . . " were human-coded on a set of approximately 100 messages, giving a recall of 0.70, precision of 0.72, and Kappa of 0.52.

The researchers used probit regressions to predict the whether a message got a reply (1 or 0) and whether the poster would return.

Features

Newsgroup level:

• Traffic on focal day
• Newsgroup type (hobby, political, technical, or health support)

Individual level:

• Whether the poster was a newcomer
• Whether the replier was a newcomer (used in post-again analysis)

Message level:

• Whether the message was cross-posted to multiple groups
• How on-topic the language in the message was to the group (calculated using average document frequency)
• Whether the message contained a personal introduction
• Whether the message contained a question
• Linguistic complexity (average sentence and word length -- of the message and any replies it received)
• Pronouns

Results

Getting a reply: Overall, about 73% of the sampled messages got replies, and the newsgroup, poster status, and message text had a large impact on reply likelihood. Newcomers were about 4% less likely to get a reply than were members who had posted at least once prior in the group. Personal introductions increased the likelihood of reply by about 10%, questions by about 6%, and being on-topic by about 10%. Messages with long sentences and big words were less likely to get replies. Pronouns also affected reply rates: First person singular pronouns (I/me/my) led to more replies, over and above the effect from personal introductions.

Posting again: Approximately 50% of posters returned to post at least once more. Those who were newcomers in the sampled post were much less likely than old-timers to post again. Getting a reply increased posters' probability of posting again by about 6%, though it mattered somewhat who the reply came from. Replies from newcomers or people using long sentences slightly decreased the likelihood that a poster would return.