acl acl2011 acl2011-136 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Myle Ott ; Yejin Choi ; Claire Cardie ; Jeffrey T. Hancock
Abstract: Consumers increasingly rate, review and research products online (Jansen, 2010; Litvin et al., 2008). Consequently, websites containing consumer reviews are becoming targets of opinion spam. While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam—fictitious opinions that have been deliberately written to sound authentic. Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing.
Reference: text
sentIndex sentText sentNum sentScore
1 Consequently, websites containing consumer reviews are becoming targets of opinion spam. [sent-7, score-0.231]
2 While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam—fictitious opinions that have been deliberately written to sound authentic. [sent-8, score-1.11]
3 Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. [sent-9, score-1.362]
4 Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing. [sent-10, score-0.893]
5 1 Introduction With the ever-increasing popularity of review websites that feature user-generated opinions (e. [sent-11, score-0.169]
6 , TripAdvisor1 and Yelp2), there comes an increasing potential for monetary gain through opinion spam— inappropriate or fraudulent reviews. [sent-13, score-0.167]
7 Opinion spam can range from annoying self-promotion of an unrelated website or blog to deliberate review fraud, as in the recent case3 of a Belkin employee who 1http : / /t ripadvi s or . [sent-14, score-0.348]
8 4 While other kinds of spam have received considerable computational attention, regrettably there has been little work to date (see Section 2) on opinion spam detection. [sent-20, score-0.742]
9 Furthermore, most previous work in the area has focused on the detection of DISRUPTIVE OPINION SPAM—uncontroversial instances of spam that are easily identified by a human reader, e. [sent-21, score-0.36]
10 And while the presence of disruptive opinion spam is certainly a nuisance, the risk it poses to the user is minimal, since the user can always choose to ignore it. [sent-24, score-0.465]
11 We focus here on a potentially ous type of opinion spam: SPAM—fictitious opinions more DECEPTIVE insidi- OPINION that have been deliber- ately written to sound authentic, in order to deceive the reader. [sent-25, score-0.267]
12 For example, one of the following two hotel reviews is truthful and the other is deceptive opinion spam: 1. [sent-26, score-1.298]
13 We will definatly be 4It is also possible for opinion spam to be negative, potentially in order to sully the reputation of a competitor. [sent-40, score-0.446]
14 Typically, these deceptive opinions are neither easily ignored nor even identifiable by a human reader;5 consequently, there are few good sources of labeled data for this research. [sent-44, score-0.831]
15 In contrast, one contribution of the work presented here is the creation of the first largescale, publicly available6 dataset for deceptive opinion spam research, containing 400 truthful and 400 gold-standard deceptive reviews. [sent-46, score-2.126]
16 To obtain a deeper understanding of the nature of deceptive opinion spam, we explore the relative utility of three potentially complementary framings of our problem. [sent-47, score-0.843]
17 , 2003); and, (c) a problem of genre identification, in which we view deceptive and truthful writing as sub-genres of imaginative and informative writing, respectively (Biber et al. [sent-50, score-1.109]
18 Particularly, we find that machine learning classifiers trained on features traditionally employed in (a) psychological studies of deception and (b) genre identification are both outperformed at statistically significant levels by ngram–based text categorization techniques. [sent-54, score-0.531]
19 Notably, a combined classifier with both n-gram and psychological deception features achieves nearly 90% cross-validated accuracy on this task. [sent-55, score-0.459]
20 In contrast, we find deceptive opinion spam detection to be well beyond the capabilities of most human judges, who perform roughly at-chance—a finding that is consistent with decades of traditional deception detection research (Bond and DePaulo, 2006). [sent-56, score-1.635]
21 5The second example review is deceptive opinion spam. [sent-57, score-0.895]
22 Specifically, we shed light on an ongoing debate in the deception literature regarding the importance of considering the context and motivation of a deception, rather than simply identifying a universal set of deception cues. [sent-62, score-0.8]
23 Lastly, our study of deceptive opinion spam detection as a genre identification problem reveals relationships between deceptive opinions and imaginative writing, and between truthful opinions and informative writing. [sent-65, score-2.524]
24 Recently, researchers have began to look at opinion spam as well (Jindal and Liu, 2008; Wu et al. [sent-71, score-0.446]
25 Jindal and Liu (2008) find that opinion spam is both widespread and different in nature from either e-mail or Web spam. [sent-73, score-0.446]
26 Using product review data, and in the absence of gold-standard deceptive opinions, they train models using features based on the review text, reviewer, and product, to distinguish between duplicate opinions7 (considered deceptive spam) and non-duplicate opinions (considered truthful). [sent-74, score-1.607]
27 (2010) propose an alternative strategy for detecting deceptive opinion spam in the absence 7Duplicate (or near-duplicate) opinions are opinions that appear more than once in the corpus with the same (or similar) text. [sent-76, score-1.407]
28 While these opinions are likely to be deceptive, they are unlikely to be representative of deceptive opinion spam in general. [sent-77, score-1.256]
29 Both of these heuristic evaluation approaches are unnecessary in our work, since we compare gold-standard deceptive and truthful opinions. [sent-80, score-0.987]
30 Yoo and Gretzel (2009) gather 40 truthful and 42 deceptive hotel reviews and, using a standard statistical test, manually compare the psychologically relevant linguistic differences between them. [sent-81, score-1.148]
31 In contrast, we create a much larger dataset of 800 opinions that we use to develop and evaluate automated deception classifiers. [sent-82, score-0.544]
32 Research has also been conducted on the related task of psycholinguistic deception detection. [sent-83, score-0.427]
33 (2004; 2008) consider computer-mediated deception in role-playing games designed to be played over instant messaging and e-mail. [sent-89, score-0.389]
34 However, while these studies compare n-gram–based deception classifiers to a random guess baseline of 50%, we additionally evaluate and compare two other computational approaches (described in Section 4), as well as the performance of human judges (described in Section 3. [sent-90, score-0.521]
35 Unfortunately, most measures of quality employed in those works are based exclusively on human judgments, which we find in Section 3 to be poorly calibrated to detecting deceptive opinion spam. [sent-96, score-0.898]
36 3 Dataset Construction and Human Performance While truthful opinions are ubiquitous online, deceptive opinions are difficult to obtain without resorting to heuristic methods (Jindal and Liu, 2008; Wu et al. [sent-97, score-1.221]
37 In this section, we report our ef- forts to gather (and validate with human judgments) the first publicly available opinion spam dataset with gold-standard deceptive opinions. [sent-99, score-1.16]
38 311 Following the work ofYoo and Gretzel (2009), we compare truthful and deceptive positive reviews for hotels found on TripAdvisor. [sent-100, score-1.152]
39 Specifically, we mine all 5-star truthful reviews from the 20 most popular hotels on TripAdvisor8 in the Chicago area. [sent-101, score-0.459]
40 9 Deceptive opinions are gathered for those same 20 hotels using Amazon Mechanical Turk10 (AMT). [sent-102, score-0.22]
41 Below, we provide details of the collection methodologies for deceptive (Section 3. [sent-103, score-0.693]
42 Ultimately, we collect 20 truthful and 20 deceptive opinions for each of the 20 chosen hotels (800 opinions total). [sent-106, score-1.305]
43 To solicit gold-standard deceptive opinion spam using AMT, we create a pool of 400 HumanIntelligence Tasks (HITs) and allocate them evenly across our 20 chosen hotels. [sent-109, score-1.139]
44 To ensure that opinions are written by unique authors, we allow only a single submission per Turker. [sent-110, score-0.143]
45 9It has been hypothesized that popular offerings are less likely to become targets of deceptive opinion spam, since the relative impact of the spam in such cases is small (Jindal and Liu, 2008; Lim et al. [sent-117, score-1.139]
46 By considering only the most popular hotels, we hope to minimize the risk of mining opinion spam and labeling it as truthful. [sent-119, score-0.446]
47 7324 5 Table 1: Descriptive statistics for 400 deceptive opinion spam submissions gathered using AMT. [sent-127, score-1.192]
48 short,11 plagiarized,12 It took approximately 14 days to collect 400 satisfactory deceptive opinions. [sent-133, score-0.693]
49 2 Truthful opinions from TripAdvisor For truthful opinions, we mine all 6,977 reviews from the 20 most popular Chicago hotels on TripAdvisor. [sent-142, score-0.576]
50 From these we eliminate: • 3,130 non-5-star reviews; • 41 non-English reviews;13 • 75 reviews with fewer than 150 characters since, by construction, deceptive opinions are 11A submission is considered unreasonably short if it tains fewer than 150 characters. [sent-143, score-0.936]
51 1); 1,607 reviews written by first-time authors— new users ewwhso whraitvtee nno bty previously posted an opinion on TripAdvisor—since these opinions are more likely to contain opinion spam, which would reduce the integrity of our truthful review data (Wu et al. [sent-149, score-0.844]
52 Finally, we balance the number of truthful and deceptive opinions by selecting 400 of the remaining 2,124 truthful reviews, such that the document lengths of the selected truthful reviews are similarly distributed to those of the deceptive reviews. [sent-151, score-2.466]
53 Thus, for each of the 20 chosen hotels, we select 20 truthful reviews from a log-normal (lefttruncated at 150 characters) distribution fit to the lengths of the deceptive reviews. [sent-154, score-1.068]
54 14 Combined with the 400 deceptive reviews gathered in Section 3. [sent-155, score-0.793]
55 3 Human performance Assessing human deception detection performance is important for several reasons. [sent-158, score-0.453]
56 Second, assessing human performance is necessary to validate the deceptive opinions gathered in Section 3. [sent-160, score-0.874]
57 If human performance is low, then our deceptive opinions are convincing, and therefore, deserving of further attention. [sent-162, score-0.831]
58 Specifically, the MAJORITY meta-judge predicts “deceptive” when at least two out of three human judges believe the review to be deceptive, and the SKEPTIC meta-judge predicts “deceptive” when any human judge believes the review to be deceptive. [sent-181, score-0.278]
59 Furthermore, all three judges suffer from truth-bias (Vrij, 2008), a common finding in deception detection research in which human judges are more likely to classify an opinion as truthful than deceptive. [sent-188, score-1.063]
60 We suspect that agreement among our human judges is so low precisely because humans are poor judges of deception (Vrij, 2008), and therefore they perform nearly at-chance respective to one another. [sent-200, score-0.614]
61 4 Automated Approaches to Deceptive Opinion Spam Detection We consider three automated approaches to detecting deceptive opinion spam, each of which utilizes classifiers (described in Section 4. [sent-201, score-0.943]
62 In our genre identification approach to deceptive opinion spam detection, we test if such a relationship exists for truthful and deceptive reviews by constructing, for each review, features based on the frequencies of each POS tag. [sent-208, score-2.262]
63 2 Psycholinguistic deception detection The Linguistic Inquiry and Word Count (LIWC) software (Pennebaker et al. [sent-211, score-0.432]
64 , 2010), and, most relevantly, to analyze deception (Hancock et al. [sent-216, score-0.389]
65 While other features have been considered in past deception detection work, notably those of Zhou et al. [sent-231, score-0.432]
66 Thus, we focus our psycholinguistic approach to deception detection on LIWC-based features. [sent-234, score-0.47]
67 3 Text categorization In contrast to the other strategies just discussed, our text categorization approach to deception detection allows us to model both content and context with n-gram features. [sent-236, score-0.488]
68 (2008), we use the SRI Language Modeling Toolkit (Stolcke, 2002) to estimate individual language models, Pr( x~ | y = c), mfora etru inthdfiuvli aunadl deceptive opinions. [sent-244, score-0.693]
69 5 Results and Discussion The deception detection strategies described in Section 4 are evaluated using a 5-fold nested crossvalidation (CV) procedure (Quadrianto et al. [sent-256, score-0.432]
70 Folds are selected so that each contains all reviews from four hotels; thus, learned models are always evaluated on reviews from unseen hotels. [sent-258, score-0.179]
71 We observe that automated classifiers outperform human judges for every metric, except truthful recall where JUDGE 2 performs best. [sent-260, score-0.464]
72 16 However, this is expected given that untrained humans often focus on unreliable cues to deception (Vrij, 2008). [sent-261, score-0.412]
73 For example, one study examining deception in online dating found that humans perform at-chance detecting deceptive profiles because they rely on text-based cues that are unrelated to deception, such as second-person pronouns (Toma and Hancock, In Press). [sent-262, score-1.176]
74 While achieving 95% truthful recall, this judge’s corresponding precision was not significantly better than chance (two-tailed binomial p = 0. [sent-267, score-0.294]
75 315 mated classifier outperforms most human judges (one-tailed sign test p = 0. [sent-269, score-0.139]
76 This result is best explained by theories of reality monitoring (Johnson and Raye, 1981), which suggest that truthful and deceptive opinions might be classified into informative and imaginative genres, respectively. [sent-273, score-1.207]
77 However, that deceptive opin- ions contain more superlatives is not unexpected, since deceptive writing (but not necessarily imaginative writing in general) often contains exaggerated language (Buller and Burgoon, 1996; Hancock et al. [sent-278, score-1.498]
78 Both remaining automated approaches to detecting deceptive opinion spam outperform the simple 17Past participle verbs were an exception. [sent-280, score-1.211]
79 (2001), we expect weights on the left to be positive (predictive of truthful opinions), and weights on the right to be negative (predictive of deceptive opinions). [sent-284, score-0.987]
80 This suggests that a universal set of keyword-based deception cues (e. [sent-300, score-0.434]
81 , BIGRAMS+) might be necessary to achieve state-of-the-art deception detection performance. [sent-304, score-0.432]
82 In agreement with theories of reality monitoring (Johnson and Raye, 1981), we observe that truthful opinions tend to include more sensorial and concrete language than deceptive opinions; in 19The result is not significantly better than BIGRAMSS+VM. [sent-306, score-1.158]
83 316 TRULTIHWFCU+LBIGRDAEMCSES+PVTMIVETRUTHFULLIWCDSVEMCEPTIVE Table 5: Top 15 highest weighted truthful and deceptive features learned by LIWC+BIGRAMSS+VM and LIWCSVM. [sent-307, score-1.004]
84 particular, truthful opinions are more specific about spatial configurations (e. [sent-312, score-0.436]
85 Accordingly, we observe an increased focus in deceptive opinions on aspects external to the hotel being reviewed (e. [sent-317, score-0.89]
86 We also acknowledge several findings that, on the surface, are in contrast to previous psycholinguistic studies of deception (Hancock et al. [sent-320, score-0.427]
87 For instance, while deception is often associated with negative emotion terms, our deceptive reviews have more positive and fewer negative emotion terms. [sent-323, score-1.163]
88 Additional work is required, but these findings further suggest the importance of moving beyond a universal set of deceptive language features (e. [sent-328, score-0.715]
89 , BIGRAMS+) and motivational parameters underlying a deception as well. [sent-332, score-0.389]
90 6 Conclusion and Future Work In this work we have developed the first large-scale dataset containing gold-standard deceptive opinion spam. [sent-333, score-0.843]
91 With it, we have shown that the detection of deceptive opinion spam is well beyond the capabilities of human judges, most of whom perform roughly at-chance. [sent-334, score-1.203]
92 Accordingly, we have introduced three automated approaches to deceptive opinion spam detection, based on insights coming from research in computational linguistics and psychology. [sent-335, score-1.177]
93 , BIGRAMS+) and motivations underlying a deception, rather than strictly adhering to a universal set of deception cues (e. [sent-340, score-0.434]
94 Lastly, we have discovered a plausible relationship between deceptive opinion spam and imaginative writing, based on POS distributional similarities. [sent-344, score-1.205]
95 Many additional approaches to detecting deceptive opinion spam are also possible, and a focus on approaches with high deceptive precision might be useful for production environments. [sent-346, score-1.866]
96 How opinions are received by online communities: a case study on amazon. [sent-407, score-0.135]
97 On lying and being lied to: A linguistic analysis of deception in computer-mediated communication. [sent-441, score-0.409]
98 The lie detector: Explorations in the automatic recognition of deceptive language. [sent-537, score-0.693]
99 Cues to deception and ability to detect lies as a function of police interview styles. [sent-647, score-0.389]
100 A comparison of classification methods for predicting deception in computermediated communication. [sent-706, score-0.389]
wordName wordTfidf (topN-words)
[('deceptive', 0.693), ('deception', 0.389), ('spam', 0.296), ('truthful', 0.294), ('opinion', 0.15), ('liwc', 0.138), ('opinions', 0.117), ('hotels', 0.084), ('judges', 0.083), ('reviews', 0.081), ('hotel', 0.08), ('vrij', 0.076), ('jindal', 0.069), ('hancock', 0.066), ('imaginative', 0.066), ('review', 0.052), ('rayson', 0.05), ('judge', 0.049), ('detection', 0.043), ('automated', 0.038), ('bigramss', 0.038), ('gretzel', 0.038), ('psycholinguistic', 0.038), ('bigrams', 0.035), ('submissions', 0.034), ('turkers', 0.034), ('detecting', 0.034), ('liars', 0.033), ('genre', 0.033), ('psychological', 0.031), ('yoo', 0.031), ('strapparava', 0.031), ('buller', 0.028), ('skeptic', 0.028), ('tripadvisor', 0.028), ('newman', 0.028), ('classifiers', 0.028), ('categorization', 0.028), ('mihalcea', 0.026), ('submission', 0.026), ('spatial', 0.025), ('biber', 0.025), ('burgoon', 0.025), ('zhou', 0.025), ('assessing', 0.024), ('personality', 0.024), ('cues', 0.023), ('vm', 0.023), ('tourism', 0.023), ('writing', 0.023), ('universal', 0.022), ('credibility', 0.022), ('identification', 0.022), ('nearly', 0.021), ('mechanical', 0.021), ('cornell', 0.021), ('amt', 0.021), ('human', 0.021), ('chicago', 0.02), ('lying', 0.02), ('hit', 0.019), ('gathered', 0.019), ('argcmaxpr', 0.019), ('dating', 0.019), ('deceivers', 0.019), ('disruptive', 0.019), ('distancing', 0.019), ('drucker', 0.019), ('forman', 0.019), ('litvin', 0.019), ('mahony', 0.019), ('monitoring', 0.019), ('myleott', 0.019), ('ntoulas', 0.019), ('possvm', 0.019), ('raye', 0.019), ('rigby', 0.019), ('serrano', 0.019), ('toma', 0.019), ('unreasonably', 0.019), ('weerkamp', 0.019), ('weimer', 0.019), ('pennebaker', 0.019), ('fold', 0.019), ('online', 0.018), ('classifier', 0.018), ('reality', 0.018), ('wu', 0.018), ('liu', 0.017), ('agreement', 0.017), ('sign', 0.017), ('learned', 0.017), ('indeed', 0.017), ('helpfulness', 0.017), ('monetary', 0.017), ('akkaya', 0.017), ('cade', 0.017), ('conrad', 0.017), ('fictitious', 0.017), ('ithaca', 0.017)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 136 acl-2011-Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Author: Myle Ott ; Yejin Choi ; Claire Cardie ; Jeffrey T. Hancock
Abstract: Consumers increasingly rate, review and research products online (Jansen, 2010; Litvin et al., 2008). Consequently, websites containing consumer reviews are becoming targets of opinion spam. While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam—fictitious opinions that have been deliberately written to sound authentic. Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing.
2 0.1397388 159 acl-2011-Identifying Noun Product Features that Imply Opinions
Author: Lei Zhang ; Bing Liu
Abstract: Identifying domain-dependent opinion words is a key problem in opinion mining and has been studied by several researchers. However, existing work has been focused on adjectives and to some extent verbs. Limited work has been done on nouns and noun phrases. In our work, we used the feature-based opinion mining model, and we found that in some domains nouns and noun phrases that indicate product features may also imply opinions. In many such cases, these nouns are not subjective but objective. Their involved sentences are also objective sentences and imply positive or negative opinions. Identifying such nouns and noun phrases and their polarities is very challenging but critical for effective opinion mining in these domains. To the best of our knowledge, this problem has not been studied in the literature. This paper proposes a method to deal with the problem. Experimental results based on real-life datasets show promising results. 1
3 0.099332243 45 acl-2011-Aspect Ranking: Identifying Important Product Aspects from Online Consumer Reviews
Author: Jianxing Yu ; Zheng-Jun Zha ; Meng Wang ; Tat-Seng Chua
Abstract: In this paper, we dedicate to the topic of aspect ranking, which aims to automatically identify important product aspects from online consumer reviews. The important aspects are identified according to two observations: (a) the important aspects of a product are usually commented by a large number of consumers; and (b) consumers’ opinions on the important aspects greatly influence their overall opinions on the product. In particular, given consumer reviews of a product, we first identify the product aspects by a shallow dependency parser and determine consumers’ opinions on these aspects via a sentiment classifier. We then develop an aspect ranking algorithm to identify the important aspects by simultaneously considering the aspect frequency and the influence of consumers’ opinions given to each aspect on their overall opinions. The experimental results on 11 popular products in four domains demonstrate the effectiveness of our approach. We further apply the aspect ranking results to the application ofdocumentlevel sentiment classification, and improve the performance significantly.
4 0.084943607 211 acl-2011-Liars and Saviors in a Sentiment Annotated Corpus of Comments to Political Debates
Author: Paula Carvalho ; Luis Sarmento ; Jorge Teixeira ; Mario J. Silva
Abstract: We investigate the expression of opinions about human entities in user-generated content (UGC). A set of 2,800 online news comments (8,000 sentences) was manually annotated, following a rich annotation scheme designed for this purpose. We conclude that the challenge in performing opinion mining in such type of content is correctly identifying the positive opinions, because (i) they are much less frequent than negative opinions and (ii) they are particularly exposed to verbal irony. We also show that the recognition of human targets poses additional challenges on mining opinions from UGC, since they are frequently mentioned by pronouns, definite descriptions and nicknames. 1
5 0.07001058 131 acl-2011-Extracting Opinion Expressions and Their Polarities - Exploration of Pipelines and Joint Models
Author: Richard Johansson ; Alessandro Moschitti
Abstract: We investigate systems that identify opinion expressions and assigns polarities to the extracted expressions. In particular, we demonstrate the benefit of integrating opinion extraction and polarity classification into a joint model using features reflecting the global polarity structure. The model is trained using large-margin structured prediction methods. The system is evaluated on the MPQA opinion corpus, where we compare it to the only previously published end-to-end system for opinion expression extraction and polarity classification. The results show an improvement of between 10 and 15 absolute points in F-measure.
6 0.060924247 21 acl-2011-A Pilot Study of Opinion Summarization in Conversations
7 0.0604579 160 acl-2011-Identifying Sarcasm in Twitter: A Closer Look
8 0.055947486 55 acl-2011-Automatically Predicting Peer-Review Helpfulness
9 0.051160101 332 acl-2011-Using Multiple Sources to Construct a Sentiment Sensitive Thesaurus for Cross-Domain Sentiment Classification
10 0.051116534 47 acl-2011-Automatic Assessment of Coverage Quality in Intelligence Reports
11 0.04388516 82 acl-2011-Content Models with Attitude
12 0.043393418 302 acl-2011-They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems
13 0.04077195 204 acl-2011-Learning Word Vectors for Sentiment Analysis
14 0.039721109 194 acl-2011-Language Use: What can it tell us?
15 0.037258692 64 acl-2011-C-Feel-It: A Sentiment Analyzer for Micro-blogs
16 0.035773974 183 acl-2011-Joint Bilingual Sentiment Classification with Unlabeled Parallel Corpora
17 0.035114232 133 acl-2011-Extracting Social Power Relationships from Natural Language
18 0.033073943 281 acl-2011-Sentiment Analysis of Citations using Sentence Structure-Based Features
19 0.032360267 288 acl-2011-Subjective Natural Language Problems: Motivations, Applications, Characterizations, and Implications
20 0.029443413 76 acl-2011-Comparative News Summarization Using Linear Programming
topicId topicWeight
[(0, 0.084), (1, 0.072), (2, 0.036), (3, -0.003), (4, -0.017), (5, 0.021), (6, 0.011), (7, 0.004), (8, -0.003), (9, -0.026), (10, -0.018), (11, -0.013), (12, -0.032), (13, 0.002), (14, -0.019), (15, 0.027), (16, -0.002), (17, -0.018), (18, 0.003), (19, 0.009), (20, -0.007), (21, 0.043), (22, -0.027), (23, 0.03), (24, -0.005), (25, -0.01), (26, 0.128), (27, -0.05), (28, -0.034), (29, 0.077), (30, 0.165), (31, -0.02), (32, -0.104), (33, 0.07), (34, 0.02), (35, 0.024), (36, -0.034), (37, 0.037), (38, 0.035), (39, -0.03), (40, -0.061), (41, 0.026), (42, 0.019), (43, -0.036), (44, 0.01), (45, 0.017), (46, -0.0), (47, 0.04), (48, 0.01), (49, 0.074)]
simIndex simValue paperId paperTitle
same-paper 1 0.92219645 136 acl-2011-Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Author: Myle Ott ; Yejin Choi ; Claire Cardie ; Jeffrey T. Hancock
Abstract: Consumers increasingly rate, review and research products online (Jansen, 2010; Litvin et al., 2008). Consequently, websites containing consumer reviews are becoming targets of opinion spam. While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam—fictitious opinions that have been deliberately written to sound authentic. Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing.
2 0.84941691 159 acl-2011-Identifying Noun Product Features that Imply Opinions
Author: Lei Zhang ; Bing Liu
Abstract: Identifying domain-dependent opinion words is a key problem in opinion mining and has been studied by several researchers. However, existing work has been focused on adjectives and to some extent verbs. Limited work has been done on nouns and noun phrases. In our work, we used the feature-based opinion mining model, and we found that in some domains nouns and noun phrases that indicate product features may also imply opinions. In many such cases, these nouns are not subjective but objective. Their involved sentences are also objective sentences and imply positive or negative opinions. Identifying such nouns and noun phrases and their polarities is very challenging but critical for effective opinion mining in these domains. To the best of our knowledge, this problem has not been studied in the literature. This paper proposes a method to deal with the problem. Experimental results based on real-life datasets show promising results. 1
3 0.79473251 211 acl-2011-Liars and Saviors in a Sentiment Annotated Corpus of Comments to Political Debates
Author: Paula Carvalho ; Luis Sarmento ; Jorge Teixeira ; Mario J. Silva
Abstract: We investigate the expression of opinions about human entities in user-generated content (UGC). A set of 2,800 online news comments (8,000 sentences) was manually annotated, following a rich annotation scheme designed for this purpose. We conclude that the challenge in performing opinion mining in such type of content is correctly identifying the positive opinions, because (i) they are much less frequent than negative opinions and (ii) they are particularly exposed to verbal irony. We also show that the recognition of human targets poses additional challenges on mining opinions from UGC, since they are frequently mentioned by pronouns, definite descriptions and nicknames. 1
4 0.78730732 131 acl-2011-Extracting Opinion Expressions and Their Polarities - Exploration of Pipelines and Joint Models
Author: Richard Johansson ; Alessandro Moschitti
Abstract: We investigate systems that identify opinion expressions and assigns polarities to the extracted expressions. In particular, we demonstrate the benefit of integrating opinion extraction and polarity classification into a joint model using features reflecting the global polarity structure. The model is trained using large-margin structured prediction methods. The system is evaluated on the MPQA opinion corpus, where we compare it to the only previously published end-to-end system for opinion expression extraction and polarity classification. The results show an improvement of between 10 and 15 absolute points in F-measure.
5 0.58865452 45 acl-2011-Aspect Ranking: Identifying Important Product Aspects from Online Consumer Reviews
Author: Jianxing Yu ; Zheng-Jun Zha ; Meng Wang ; Tat-Seng Chua
Abstract: In this paper, we dedicate to the topic of aspect ranking, which aims to automatically identify important product aspects from online consumer reviews. The important aspects are identified according to two observations: (a) the important aspects of a product are usually commented by a large number of consumers; and (b) consumers’ opinions on the important aspects greatly influence their overall opinions on the product. In particular, given consumer reviews of a product, we first identify the product aspects by a shallow dependency parser and determine consumers’ opinions on these aspects via a sentiment classifier. We then develop an aspect ranking algorithm to identify the important aspects by simultaneously considering the aspect frequency and the influence of consumers’ opinions given to each aspect on their overall opinions. The experimental results on 11 popular products in four domains demonstrate the effectiveness of our approach. We further apply the aspect ranking results to the application ofdocumentlevel sentiment classification, and improve the performance significantly.
6 0.54164511 288 acl-2011-Subjective Natural Language Problems: Motivations, Applications, Characterizations, and Implications
7 0.51761824 21 acl-2011-A Pilot Study of Opinion Summarization in Conversations
8 0.50531137 55 acl-2011-Automatically Predicting Peer-Review Helpfulness
9 0.44236699 297 acl-2011-That's What She Said: Double Entendre Identification
10 0.43387777 84 acl-2011-Contrasting Opposing Views of News Articles on Contentious Issues
11 0.43198335 194 acl-2011-Language Use: What can it tell us?
12 0.42936605 156 acl-2011-IMASS: An Intelligent Microblog Analysis and Summarization System
13 0.40457949 82 acl-2011-Content Models with Attitude
14 0.36688578 77 acl-2011-Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech
15 0.36133927 301 acl-2011-The impact of language models and loss functions on repair disfluency detection
16 0.34165844 162 acl-2011-Identifying the Semantic Orientation of Foreign Words
17 0.34032187 20 acl-2011-A New Dataset and Method for Automatically Grading ESOL Texts
18 0.32146433 133 acl-2011-Extracting Social Power Relationships from Natural Language
19 0.31503579 248 acl-2011-Predicting Clicks in a Vocabulary Learning System
20 0.31346607 289 acl-2011-Subjectivity and Sentiment Analysis of Modern Standard Arabic
topicId topicWeight
[(5, 0.03), (11, 0.272), (17, 0.037), (26, 0.032), (31, 0.016), (37, 0.069), (39, 0.033), (41, 0.052), (53, 0.035), (55, 0.021), (57, 0.011), (59, 0.031), (72, 0.065), (73, 0.017), (88, 0.014), (91, 0.03), (96, 0.106), (97, 0.012)]
simIndex simValue paperId paperTitle
same-paper 1 0.7354598 136 acl-2011-Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Author: Myle Ott ; Yejin Choi ; Claire Cardie ; Jeffrey T. Hancock
Abstract: Consumers increasingly rate, review and research products online (Jansen, 2010; Litvin et al., 2008). Consequently, websites containing consumer reviews are becoming targets of opinion spam. While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam—fictitious opinions that have been deliberately written to sound authentic. Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing.
2 0.66274905 48 acl-2011-Automatic Detection and Correction of Errors in Dependency Treebanks
Author: Alexander Volokh ; Gunter Neumann
Abstract: Annotated corpora are essential for almost all NLP applications. Whereas they are expected to be of a very high quality because of their importance for the followup developments, they still contain a considerable number of errors. With this work we want to draw attention to this fact. Additionally, we try to estimate the amount of errors and propose a method for their automatic correction. Whereas our approach is able to find only a portion of the errors that we suppose are contained in almost any annotated corpus due to the nature of the process of its creation, it has a very high precision, and thus is in any case beneficial for the quality of the corpus it is applied to. At last, we compare it to a different method for error detection in treebanks and find out that the errors that we are able to detect are mostly different and that our approaches are complementary. 1
3 0.57278746 71 acl-2011-Coherent Citation-Based Summarization of Scientific Papers
Author: Amjad Abu-Jbara ; Dragomir Radev
Abstract: In citation-based summarization, text written by several researchers is leveraged to identify the important aspects of a target paper. Previous work on this problem focused almost exclusively on its extraction aspect (i.e. selecting a representative set of citation sentences that highlight the contribution of the target paper). Meanwhile, the fluency of the produced summaries has been mostly ignored. For example, diversity, readability, cohesion, and ordering of the sentences included in the summary have not been thoroughly considered. This resulted in noisy and confusing summaries. In this work, we present an approach for producing readable and cohesive citation-based summaries. Our experiments show that the pro- posed approach outperforms several baselines in terms of both extraction quality and fluency.
4 0.55828655 318 acl-2011-Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models
Author: Jason Naradowsky ; Kristina Toutanova
Abstract: This paper describes an unsupervised dynamic graphical model for morphological segmentation and bilingual morpheme alignment for statistical machine translation. The model extends Hidden Semi-Markov chain models by using factored output nodes and special structures for its conditional probability distributions. It relies on morpho-syntactic and lexical source-side information (part-of-speech, morphological segmentation) while learning a morpheme segmentation over the target language. Our model outperforms a competitive word alignment system in alignment quality. Used in a monolingual morphological segmentation setting it substantially improves accuracy over previous state-of-the-art models on three Arabic and Hebrew datasets.
5 0.52738619 183 acl-2011-Joint Bilingual Sentiment Classification with Unlabeled Parallel Corpora
Author: Bin Lu ; Chenhao Tan ; Claire Cardie ; Benjamin K. Tsou
Abstract: Most previous work on multilingual sentiment analysis has focused on methods to adapt sentiment resources from resource-rich languages to resource-poor languages. We present a novel approach for joint bilingual sentiment classification at the sentence level that augments available labeled data in each language with unlabeled parallel data. We rely on the intuition that the sentiment labels for parallel sentences should be similar and present a model that jointly learns improved monolingual sentiment classifiers for each language. Experiments on multiple data sets show that the proposed approach (1) outperforms the monolingual baselines, significantly improving the accuracy for both languages by 3.44%-8. 12%; (2) outperforms two standard approaches for leveraging unlabeled data; and (3) produces (albeit smaller) performance gains when employing pseudo-parallel data from machine translation engines. 1
6 0.52127099 330 acl-2011-Using Derivation Trees for Treebank Error Detection
7 0.51147413 32 acl-2011-Algorithm Selection and Model Adaptation for ESL Correction Tasks
8 0.50739878 88 acl-2011-Creating a manually error-tagged and shallow-parsed learner corpus
9 0.50654715 160 acl-2011-Identifying Sarcasm in Twitter: A Closer Look
10 0.50528258 261 acl-2011-Recognizing Named Entities in Tweets
11 0.50323433 246 acl-2011-Piggyback: Using Search Engines for Robust Cross-Domain Named Entity Recognition
12 0.50208622 37 acl-2011-An Empirical Evaluation of Data-Driven Paraphrase Generation Techniques
13 0.5018363 252 acl-2011-Prototyping virtual instructors from human-human corpora
14 0.50106972 327 acl-2011-Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
15 0.49985582 119 acl-2011-Evaluating the Impact of Coder Errors on Active Learning
16 0.49905148 87 acl-2011-Corpus Expansion for Statistical Machine Translation with Semantic Role Label Substitution Rules
17 0.49885634 108 acl-2011-EdIt: A Broad-Coverage Grammar Checker Using Pattern Grammar
18 0.49864465 132 acl-2011-Extracting Paraphrases from Definition Sentences on the Web
19 0.49741644 64 acl-2011-C-Feel-It: A Sentiment Analyzer for Micro-blogs
20 0.49720162 40 acl-2011-An Error Analysis of Relation Extraction in Social Media Documents