Yi et al, ICDM 2003
From ScribbleWiki: Analysis of Social Media
Sentiment Analyzer: Extracting Sentiments about a Given Topic using Natural Language Processing Techniques.
It's one of the early papers that extends previous work on document-level sentiment classification to more fine-grained topic-specific sentiment extraction. The tool that the paper presents is called Sentiment Analyzer (SA). Given a topic, SA detects all relevant feature terms, and determines sentiment for each reference to the feature.
Basically SA works in 3 steps:
1. Topic specific feature term extraction:
A feature term is defined to be a term that's either a part/attribute of the given topic, or an attribute of a known feature (feature's feature, e.g. battery life for the 'digital camera' domain). To do this, SA first extract all the noun phrases from a document, and then apply feature selection to these candidates.
For noun phrases, they experimented with 3 extraction patterns: Base Noun Phrases (BNP), Definite Base Noun Phrases (DBNP), and Beginning Definite Base Noun Phrases (bDBNP).
For feature selection, the paper tested two algorithms: mixture language model and likelihood ratio. The first one models the language in any document as a mixture (or a linear combination) of the general web language model θW and a topic-specific language model θT (T denotes the topic). Assuming a multinomial distribution for θW and θT, both the model parameters and the weights can be derived by maximum likelihood estimation. Feature terms are just those with high θTi scores. The latter algorithm is based on likelihood-ratio test by Dunning. It is computed for each candidate feature term, according to the likelihood of the feature term occuring in documents focused on topic T and in documents not focused on topic T. The higher the likelihood ratio, the more likely it's a feature term for topic T.
For evaluation of this part, they extracted 38 and 31 feature terms respectively for digital camera and music review domain. Manual inspection shows that likelihood ratio performs much better than mixture language model for feature selection; and bDBNP candidate extracton combined with likehood ratio test achieves the best accuracy, up to 97% for digital camera domain, and 100% for music domain. (However, given that the feature set is quite small, and most of them are easily identifiable, it's difficult to see the advantage of statistical learning over manual design).
After collecting the feature terms for a given topic at this step, only those sentence containing mentions of these feature terms are extracted from the input document, as inputs to the sentiment analysis at the next step.
2. sentiment extraction
For this task, SA simply utilizes a sentiment lexicon, which contains lists of words together with their POS and sentiment category (positive and negative). The lexicon is collected from General Inquirer, Dictionary of Affect of Language, and WordNet.
With the sentiment lexicon (and a syntactic parser), SA is able to identify all the sentiment phrases for each input sentence. A sentiment phrase can be an adjective phrase with the adjective defined in the sentiment lexicon, or an subject, object and prepositional phrases that contains at least one sentiment word. The sentiment of the phrase is simply the sentiment of the corresponding word, reversed when a negative word is present.
3. (subject, sentiment) association
SA draws on another linguistic resources at this step: the sentiment pattern database. It contains sentiment extraction patterns (sentiment category and target) for sentence predicates, constructed and manually refined from the same resources as the sentiment lexicon.
These sentiment patterns are directly used for assigning the polarity of each sentiment phrase to its target. At a sentence level, SA first extract the certain types of ternary expressions (T-expressions) that look like <target, verb, ""> or <target, verb, source>. For sentences with no verbs(text fragment), binary expressions (B-expressions, with the form <adjective, target>) are extracted instead. Then these expressions are used to make (subject, sentiment) association. Specifically, the T or B expressions of a input sentence is matched against the sentiment phrases (with the target already identified). Once a matching phrase is found, the target and sentiment assignment are simply determined as defined for that phrase.
The paper evaluated SA against ReviewSeer (Dave et al., 2003), which is claimed to be "by far the latest and the best opinion classifier" at that time. For experiments on product review dataset, SA was outperformed by ReviewSeer in accuracy (85.6% vs. 88.4%)(though the results are not directly comparable because of difference in dataset). In the task of classifying general web documents, however, SA achieves much higher accuracy(90% ～ 93%) than ReviewSeer (only 38%). It's because for a statitical document-level classifier, enough sentiment-bearing expressions is essential. However, these sentiment expressions are typically very sparse in general web documents; actually most of the web documents are neutral.
The sentiment analysis techniques used in this paper (syntactic analysis and manually-designed pattern matching) are not very exciting on its own. However, the novel idea of classifying sentiments at a topic-feature level has been applied to many other later work, such as Minqing Hu and Bing Liu, KDD 2004 and Popescu and Etzioni, HLT-EMNLP 2005. The three stages of feature extraction--sentiment classification--sentiment association && summarization have also become standard steps of doing feature-specific sentiment analysis.