acl acl2010 acl2010-208 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Cigdem Toprak ; Niklas Jakob ; Iryna Gurevych
Abstract: In this paper, we introduce a corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information. We present a two-level annotation scheme. In the first stage, the reviews are analyzed at the sentence level for (i) relevancy to a given topic, and (ii) expressing an evaluation about the topic. In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, intensity), and the functional components of the evaluations (opinion terms, targets and holders). We discuss the annotation scheme, the inter-annotator agreement for different subtasks and our observations.
Reference: text
sentIndex sentText sentNum sentScore
1 In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, intensity), and the functional components of the evaluations (opinion terms, targets and holders). [sent-7, score-0.774]
2 Opinion mining spans a variety of subtasks including: creating opinion word lexicons (Esuli and Sebastiani, 2006; Ding et al. [sent-10, score-0.496]
3 , 2008), identifying opinion expressions (Riloff and Wiebe, 2003; Fahrni and Klenner, 2008), identifying polarities of opinions in context (Breck et al. [sent-11, score-0.6]
4 , 2005), extracting opinion targets (Hu and Liu, 2004; Zhuang et al. [sent-13, score-0.542]
5 , 2006; Cheng and Xu, 2008) and opinion holders (Kim and Hovy, 2006; Choi et al. [sent-14, score-0.483]
6 Data-driven approaches for extracting opinion expressions, their holders and targets require reliably annotated data at the expression level. [sent-16, score-0.789]
7 In previous research, expression level annotation of opinions was extensively investigated on newspaper articles (Wiebe et al. [sent-17, score-0.503]
8 To the best of our knowledge, there are two corpora of user-generated discourse which are annotated for opinion related information at the expression level: The corpus of Hu & Liu (2004) consists of customer reviews about consumer electronics, and the corpus of Zhuang et al. [sent-23, score-0.87]
9 Our goal is to create sentence and expression level annotated corpus of customer reviews which fulfills the following requirements: (1) It filters individual sentences regarding their topic rele- vancy and the existence of an opinion or factual information which implies an evaluation. [sent-27, score-1.015]
10 (2) It identifies opinion expressions including the respective opinion target, opinion holder, modifiers, and anaphoric expressions if applicable. [sent-28, score-1.367]
11 (3) The semantic orientation of the opinion expression is identified while considering negation, and the opinion expression is linked to the respective holder and target in the discourse. [sent-29, score-1.291]
12 Such a resource would (i) enable novel applications ofopinion mining such as a fine-grained identification of opinion properties, e. [sent-30, score-0.399]
13 opinion modification detection including negation, and (ii) enhance opinion target extraction and the polarity assignment by linking the opinion expression with its target 1http : / /blog . [sent-32, score-1.666]
14 1 Newspaper Articles and Meeting Dialogs Most prominent work concerning the expression level annotation of opinions is the MultiPerspective Question Answering (MPQA) corpus2 (Wiebe et al. [sent-43, score-0.47]
15 It was extended several times over the last years, either by adding new documents or annotating new types of opinion related information (Wilson and Wiebe, 2005; Stoyanov and Cardie, 2008; Wilson, 2008b). [sent-45, score-0.399]
16 Besides subjective utterances, the AMIDA scheme contains objective polar utterances which annotates evaluations without expressing explicit opinion expressions. [sent-55, score-1.239]
17 (2008) proposes opinion frames for representing discourse level associations in meeting dialogs. [sent-57, score-0.574]
18 The opinion frames are built based on the links between targets. [sent-61, score-0.428]
19 (2008) show that opinion frames enable a coherent interpretation of the 2http : / /www . [sent-63, score-0.428]
20 They propose a scheme which first identifies and assigns categories to the opinion segments as reporting, judgment, advice, or sentiment; and then links the opinion segments with each other via rhetorical relations including contrast, correction, support, result, or continuation. [sent-70, score-0.9]
21 However, in contrast to our scheme and other schemes, instead of marking expression boundaries without any restriction they annotate an opinion segment only if it contains an opinion word from their lexicon, or if it has a rhetorical relation to another opinion segment. [sent-71, score-1.538]
22 The corpus provides only target and polarity annotations, and do not contain opinion expression or opinion modifier annotations which lead to these polarity scores. [sent-80, score-1.43]
23 The annotation scheme allows the annotation of implicit features (indicated with the the attribute [u]). [sent-81, score-0.545]
24 e, it is unclear to which mention of the feature the opinion refers to. [sent-84, score-0.399]
25 The opinion words (Oword) and their semantic orientations (Otype) are identified. [sent-90, score-0.399]
26 1 Opinion versus Polar Facts The goal of the annotation scheme is to capture the evaluations regarding the topics being discussed in the consumer reviews. [sent-98, score-0.471]
27 The evaluations in consumer reviews are either explicit expressions of opinions, or facts which imply evaluations as discussed below. [sent-99, score-0.572]
28 (2005), we view opinions in terms of their functional components, as opinion holders, e. [sent-106, score-0.553]
29 We call such objectively verifiable, but evaluative sentences polar facts. [sent-120, score-0.598]
30 opinion words, loaded with a positive or negative connotation (e. [sent-123, score-0.438]
31 However, evaluations in polar facts can only be inferred within the context of the review. [sent-127, score-0.691]
32 For instance, the targets of the implied evalution in the polar facts (2), (3) and (4) are the professors. [sent-128, score-0.765]
33 First, the sentence level scheme analyses each sentence in terms of (i) its relevancy to the overall topic of the review, and (ii) whether it contains an evaluation (an opinion or a polar fact) about the topic. [sent-131, score-1.363]
34 Upon marking an opinion expression span, the target and holder of the opinion is marked and linked to the marked opinion expression. [sent-133, score-1.653]
35 Furthermore, the expression level scheme allows assigning polarities to the marked opinion expression spans and targets of the polar facts. [sent-134, score-1.735]
36 The following attributes are used: topic relevant attribute is labeled as yes if the sentence discusses the given topic itself or its aspects, properties or features as in examples (1)(4). [sent-151, score-0.719]
37 opinionated attribute is labeled as yes if the sentence contains any explicit expressions of opinions about the given topic. [sent-153, score-0.786]
38 This attribute is presented if the topic relevant attribute has been labeled as none given or yes. [sent-154, score-0.642]
39 [Topic: University of Phoenix] polar fact attribute is labeled as yes if the sentence is a polar fact. [sent-168, score-1.464]
40 This attribute is presented if the opinionated attribute has been labeled as no. [sent-169, score-0.67]
41 Examples (2)-(4) demonstrate sentences labeled as topic relevant=yes, opinionated=no and polar fact=yes. [sent-170, score-0.762]
42 polar fact polarity attribute represents the polarity of the evaluation in a polar fact sentence. [sent-171, score-1.675]
43 The value both is intended for the polar fact sentences containing more than one evaluation with contradicting polarities. [sent-173, score-0.634]
44 At the expression level analysis, the targets of the contradicting polar fact evaluations are identified distinctly and assigned polarities of positive or negative later on. [sent-174, score-1.147]
45 Examples (9)-(1 1) demonstrate examples of polar fact sentences with different values of the attribute polar fact polarity. [sent-175, score-1.352]
46 , sentences labeled as topic relevant=yes, opinionated=yes or topic relevant=yes, opinionated=no, polar fact=yes. [sent-184, score-0.887]
47 If the sentence is opinionated, then, the aim is to mark the opinion expression span, and label its polarity and strength (i. [sent-186, score-0.846]
48 At this stage, annotators mark text spans, and are allowed to assign one of the five labels to the marked span: The polar target is used to label the targets of the evaluations implied by polar facts. [sent-190, score-1.517]
49 The is- Reference attribute labels polar targets which are anaphoric references. [sent-191, score-0.923]
50 The polar target polarity 578 Figure 2: The expression level annotation scheme attribute is used to label the polarity as positive or negative. [sent-192, score-1.605]
51 If the isReference attribute is labeled as true, then the referent attribute appears which enables the annotator to resolve the reference to its antecedent. [sent-193, score-0.563]
52 The polar target in (13), written bold, is labeled as isReference=true, polar target polarity=negative. [sent-195, score-1.257]
53 To resolve the reference, annotator first creates another polar target markable for the antecedent, namely the bold text span in (12), then, links the antecedent to the referent attribute of the polar target in (13). [sent-196, score-1.61]
54 The target annotation represents what the opinion is about. [sent-199, score-0.584]
55 Both polar targets and targets can be the topic ofthe review or different aspects, i. [sent-200, score-0.976]
56 Similar to the polar targets, the isReference attribute allows the identification of the targets which are anaphoric references and the referent attribute links them to their antecedents in the discourse. [sent-203, score-1.158]
57 The holder type represents the holder of an opinion in the discourse and is labeled in the same manner as the targets and polar targets. [sent-206, score-1.365]
58 , which affect the strength of an opinion or shift its polarity. [sent-210, score-0.466]
59 The opinionexpression annotation is used to label the opinion terms in the sentence. [sent-214, score-0.632]
60 The polarity attribute assesses the semantic orientation of the attitude, where the strength attribute marks the intensity of this attitude. [sent-218, score-0.715]
61 The polarity and strength attributes focus solely on the marked opinionexpression span, not the whole evaluation implied in the sentence. [sent-219, score-0.479]
62 We infer the polarity of the evaluation only after considering the modifier, polarity and the strength attributes together. [sent-221, score-0.476]
63 Finally, Figure 3 demonstrates all expression level markables created for an opinionated sentence and how they relate to each other. [sent-231, score-0.501]
64 579 Figure 3: Expression level annotation example 4 Annotation Study Each review has been annotated by two annotators independently according to the annotation scheme introduced above. [sent-232, score-0.55]
65 A new attribute is presented based on the decision made for the previous attribute, for instance, opinionated attribute is only presented if the topic relevant attribute is labeled as yes or none given; polar fact attribute is only presented if the opinionated attribute is labeled as no etc. [sent-275, score-2.322]
66 We calculate κ for each attribute considering only the markables which were labeled the same by both annotators in the previously required step. [sent-276, score-0.41]
67 r9756e231ment The agreement for topic relevancy shows that it is possible to label this attribute reliably. [sent-281, score-0.486]
68 The sentences labeled as topic relevant by both annotators correspond to 59% of all sentences, suggesting that people often drift offthe topic in consumer reviews. [sent-282, score-0.606]
69 On the other hand, we obtain moderate agreement levels for the opinionated and polar fact attributes. [sent-284, score-0.847]
70 62% of the topic relevant sentences were labeled as opinionated by at least one annotator, and the rest 38% constitute the topic relevant sentences labeled as not opinionated by both annotators. [sent-285, score-0.924]
71 Nonetheless, they still contain evaluations (polar facts), as 15% of the topic relevant sen580 tences were labeled as polar facts by both annotators. [sent-286, score-0.931]
72 When we merge the attributes opinionated and polar fact into a single category, we obtain κ of 0. [sent-287, score-0.801]
73 Thus, we conclude that opinion-relevant sentences, either in the form of an explicit expression of opinion or a polar fact, can be labeled reliably in consumer reviews. [sent-289, score-1.327]
74 However, there is a thin border between polar facts and explicit expressions of opinions. [sent-290, score-0.697]
75 To the best ofour knowledge, similar annotation efforts on consumer or movie reviews do not provide any agreement figures for direct comparison. [sent-291, score-0.499]
76 They calculate the sentence level κ values based on the existence of a subjective expression span in the sentence. [sent-299, score-0.426]
77 Although the task definitions, approaches and the corpora have quite disparate characteristics in both studies, we obtain comparable results when we merge opinionated and polar fact categories. [sent-300, score-0.748]
78 3 Expression Level Agreement At the expression level, annotators focus only on the sentences which were labeled as opinionated or polar fact by both annotators. [sent-302, score-1.116]
79 Annotators were instructed to mark text spans, and then, assign them the annotation types such as polar target, opinionexpression etc. [sent-303, score-0.758]
80 (2008) present a discourse level annotation study in which opinion and target spans are marked and linked with each other in a meeting transcript corpus. [sent-310, score-0.867]
81 We obtain higher agreement values for both opinion expression and target spans. [sent-330, score-0.725]
82 We attribute this to the fact that the annotators look for opinion expression and target spans within the opinionated sentences which they agreed upon. [sent-331, score-1.273]
83 Compared to the high agreement on marking target spans, we obtain lower agreement values on marking polar target spans. [sent-333, score-1.003]
84 We observe that it is easier to attribute explicit expressions of evaluations to topic relevant entities compared to attributing evaluations implied by experiences to specific topic relevant entities in the reviews. [sent-334, score-0.882]
85 We considered the overlapping target and polar target spans together in this calculation, and obtained an α value of 0. [sent-336, score-0.75]
86 For instance, upon marking a text span as a polar target or an opinionexpression, one has to label the polarity and strength. [sent-356, score-0.925]
87 e5697x4pression level We observe that the strength of the opinionexpression and the polar target polarity cannot be labeled as reliably as the polarity of the opinionexpression. [sent-362, score-1.276]
88 61% of the agreed upon polar targets were labeled as negative by both annotators. [sent-363, score-0.786]
89 This indicates that reviewers tend to report negative experiences using polar facts, probably objectively describing what has happened, but report positive experiences with explicit opinion expressions. [sent-366, score-1.109]
90 4 Discussion We analyzed the discrepancies in the annotations to gain insights about the challenges involved in various opinion related labeling tasks. [sent-370, score-0.399]
91 However, they had difficulties distinguishing between a polar fact and an opinion. [sent-380, score-0.559]
92 Sentence level annotation increases the reliability of the expression level annotation in terms of marking text spans. [sent-385, score-0.665]
93 For instance, one annotator labeled the 582 opinion expression in (24) as strong, while the other one labeled it as average. [sent-387, score-0.768]
94 5 Conclusions We presented a corpus of consumer reviews from the rateitall and eopinions websites annotated with opinion related information. [sent-393, score-0.659]
95 Existing opinion annotated user-generated corpora suffer from several limitations which result in difficulties for interpreting the experimental results and for performing error analysis. [sent-394, score-0.399]
96 To name a few, they do not explicitly link the functional components of the opinions like targets, holders, or modifiers with the opinion expression; some of them do not mark opinion expression spans, none of them resolves anaphoric references in discourse. [sent-395, score-1.198]
97 Therefore, we introduced a two level annotation scheme consisting of the sentence and expression levels, which overcomes the limitations of the existing review corpora. [sent-396, score-0.538]
98 The expression level annotation further investigates on-topic sentences containing evaluations for pinpointing the properties (polarity, strength), and marking the functional components of the evaluations (opinion terms, modifiers, targets and holders), and linking them within a discourse. [sent-399, score-0.846]
99 We applied the annotation scheme to the consumer review genre and presented an extensive inter-annotator study providing insights to the challenges involved in various opinion related labeling tasks in consumer reviews. [sent-400, score-0.892]
100 Similar to the MPQA scheme, which is successfully applied to the newspaper genre, the annotation scheme treats opinions and evaluations as a composition of functional components and it is eas- ily extendable. [sent-401, score-0.508]
wordName wordTfidf (topN-words)
[('polar', 0.525), ('opinion', 0.399), ('attribute', 0.201), ('opinionated', 0.189), ('polarity', 0.178), ('expression', 0.163), ('targets', 0.143), ('topic', 0.125), ('annotation', 0.121), ('consumer', 0.115), ('opinions', 0.113), ('opinionexpression', 0.112), ('scheme', 0.102), ('somasundaran', 0.101), ('agreement', 0.099), ('evaluations', 0.098), ('spans', 0.097), ('wiebe', 0.093), ('annotators', 0.093), ('reviews', 0.089), ('wilson', 0.085), ('holders', 0.084), ('capella', 0.084), ('span', 0.082), ('labeled', 0.079), ('marking', 0.076), ('movie', 0.075), ('holder', 0.073), ('level', 0.073), ('discourse', 0.073), ('subjective', 0.069), ('facts', 0.068), ('markable', 0.067), ('strength', 0.067), ('target', 0.064), ('relevancy', 0.061), ('lenient', 0.061), ('yes', 0.061), ('expressions', 0.058), ('professors', 0.056), ('private', 0.055), ('anaphoric', 0.054), ('attributes', 0.053), ('modifier', 0.049), ('annotator', 0.048), ('explicit', 0.046), ('zhuang', 0.042), ('attitude', 0.042), ('contradicting', 0.042), ('ftype', 0.042), ('fword', 0.042), ('isreference', 0.042), ('nightmare', 0.042), ('otype', 0.042), ('oword', 0.042), ('phoenix', 0.042), ('syllabus', 0.042), ('functional', 0.041), ('theresa', 0.041), ('marked', 0.04), ('review', 0.04), ('janyce', 0.04), ('objectively', 0.04), ('mpqa', 0.04), ('negative', 0.039), ('sentence', 0.039), ('hu', 0.038), ('intensity', 0.038), ('reliability', 0.038), ('markables', 0.037), ('relevant', 0.036), ('regarding', 0.035), ('sentiment', 0.035), ('referent', 0.034), ('adt', 0.034), ('fact', 0.034), ('sentences', 0.033), ('newspaper', 0.033), ('claire', 0.033), ('customer', 0.031), ('orientation', 0.03), ('polarities', 0.03), ('experiences', 0.03), ('passonneau', 0.03), ('implied', 0.029), ('modifiers', 0.029), ('frames', 0.029), ('disagreements', 0.028), ('choi', 0.028), ('amida', 0.028), ('appauled', 0.028), ('complaints', 0.028), ('devry', 0.028), ('eopinions', 0.028), ('fulfills', 0.028), ('iment', 0.028), ('insulting', 0.028), ('nonsense', 0.028), ('pheonix', 0.028), ('rateitall', 0.028)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000008 208 acl-2010-Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
Author: Cigdem Toprak ; Niklas Jakob ; Iryna Gurevych
Abstract: In this paper, we introduce a corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information. We present a two-level annotation scheme. In the first stage, the reviews are analyzed at the sentence level for (i) relevancy to a given topic, and (ii) expressing an evaluation about the topic. In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, intensity), and the functional components of the evaluations (opinion terms, targets and holders). We discuss the annotation scheme, the inter-annotator agreement for different subtasks and our observations.
2 0.39925045 22 acl-2010-A Unified Graph Model for Sentence-Based Opinion Retrieval
Author: Binyang Li ; Lanjun Zhou ; Shi Feng ; Kam-Fai Wong
Abstract: There is a growing research interest in opinion retrieval as on-line users’ opinions are becoming more and more popular in business, social networks, etc. Practically speaking, the goal of opinion retrieval is to retrieve documents, which entail opinions or comments, relevant to a target subject specified by the user’s query. A fundamental challenge in opinion retrieval is information representation. Existing research focuses on document-based approaches and documents are represented by bag-of-word. However, due to loss of contextual information, this representation fails to capture the associative information between an opinion and its corresponding target. It cannot distinguish different degrees of a sentiment word when associated with different targets. This in turn seriously affects opinion retrieval performance. In this paper, we propose a sentence-based approach based on a new information representa- , tion, namely topic-sentiment word pair, to capture intra-sentence contextual information between an opinion and its target. Additionally, we consider inter-sentence information to capture the relationships among the opinions on the same topic. Finally, the two types of information are combined in a unified graph-based model, which can effectively rank the documents. Compared with existing approaches, experimental results on the COAE08 dataset showed that our graph-based model achieved significant improvement. 1
3 0.39082959 251 acl-2010-Using Anaphora Resolution to Improve Opinion Target Identification in Movie Reviews
Author: Niklas Jakob ; Iryna Gurevych
Abstract: unkown-abstract
4 0.35527474 134 acl-2010-Hierarchical Sequential Learning for Extracting Opinions and Their Attributes
Author: Yejin Choi ; Claire Cardie
Abstract: Automatic opinion recognition involves a number of related tasks, such as identifying the boundaries of opinion expression, determining their polarity, and determining their intensity. Although much progress has been made in this area, existing research typically treats each of the above tasks in isolation. In this paper, we apply a hierarchical parameter sharing technique using Conditional Random Fields for fine-grained opinion analysis, jointly detecting the boundaries of opinion expressions as well as determining two of their key attributes polarity and intensity. Our experimental results show that our proposed approach improves the performance over a baseline that does not — exploit hierarchical structure among the classes. In addition, we find that the joint approach outperforms a baseline that is based on cascading two separate components.
5 0.31944892 123 acl-2010-Generating Focused Topic-Specific Sentiment Lexicons
Author: Valentin Jijkoun ; Maarten de Rijke ; Wouter Weerkamp
Abstract: We present a method for automatically generating focused and accurate topicspecific subjectivity lexicons from a general purpose polarity lexicon that allow users to pin-point subjective on-topic information in a set of relevant documents. We motivate the need for such lexicons in the field of media analysis, describe a bootstrapping method for generating a topic-specific lexicon from a general purpose polarity lexicon, and evaluate the quality of the generated lexicons both manually and using a TREC Blog track test set for opinionated blog post retrieval. Although the generated lexicons can be an order of magnitude more selective than the general purpose lexicon, they maintain, or even improve, the performance of an opin- ion retrieval system.
6 0.16461152 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
7 0.13725837 141 acl-2010-Identifying Text Polarity Using Random Walks
8 0.12669182 113 acl-2010-Extraction and Approximation of Numerical Attributes from the Web
9 0.12260086 2 acl-2010-"Was It Good? It Was Provocative." Learning the Meaning of Scalar Adjectives
10 0.11338596 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
11 0.10957469 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
12 0.10868617 105 acl-2010-Evaluating Multilanguage-Comparability of Subjectivity Analysis Systems
13 0.10436736 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
14 0.084779963 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
15 0.083166584 31 acl-2010-Annotation
16 0.083152413 42 acl-2010-Automatically Generating Annotator Rationales to Improve Sentiment Classification
17 0.07828334 231 acl-2010-The Prevalence of Descriptive Referring Expressions in News and Narrative
18 0.074088998 149 acl-2010-Incorporating Extra-Linguistic Information into Reference Resolution in Collaborative Task Dialogue
19 0.073223948 33 acl-2010-Assessing the Role of Discourse References in Entailment Inference
20 0.073022515 58 acl-2010-Classification of Feedback Expressions in Multimodal Data
topicId topicWeight
[(0, -0.194), (1, 0.19), (2, -0.28), (3, 0.268), (4, -0.284), (5, 0.085), (6, -0.071), (7, 0.234), (8, 0.01), (9, -0.05), (10, 0.06), (11, -0.056), (12, 0.041), (13, 0.129), (14, 0.027), (15, -0.016), (16, -0.167), (17, 0.166), (18, 0.062), (19, 0.063), (20, 0.032), (21, 0.047), (22, 0.041), (23, 0.076), (24, -0.03), (25, -0.024), (26, -0.004), (27, 0.091), (28, 0.057), (29, 0.04), (30, -0.104), (31, 0.009), (32, 0.041), (33, -0.033), (34, 0.019), (35, 0.057), (36, 0.093), (37, 0.013), (38, 0.048), (39, 0.016), (40, 0.019), (41, -0.029), (42, -0.011), (43, -0.002), (44, -0.035), (45, -0.086), (46, 0.041), (47, 0.026), (48, 0.004), (49, 0.019)]
simIndex simValue paperId paperTitle
same-paper 1 0.96908933 208 acl-2010-Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
Author: Cigdem Toprak ; Niklas Jakob ; Iryna Gurevych
Abstract: In this paper, we introduce a corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information. We present a two-level annotation scheme. In the first stage, the reviews are analyzed at the sentence level for (i) relevancy to a given topic, and (ii) expressing an evaluation about the topic. In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, intensity), and the functional components of the evaluations (opinion terms, targets and holders). We discuss the annotation scheme, the inter-annotator agreement for different subtasks and our observations.
2 0.89604223 251 acl-2010-Using Anaphora Resolution to Improve Opinion Target Identification in Movie Reviews
Author: Niklas Jakob ; Iryna Gurevych
Abstract: unkown-abstract
3 0.88305283 134 acl-2010-Hierarchical Sequential Learning for Extracting Opinions and Their Attributes
Author: Yejin Choi ; Claire Cardie
Abstract: Automatic opinion recognition involves a number of related tasks, such as identifying the boundaries of opinion expression, determining their polarity, and determining their intensity. Although much progress has been made in this area, existing research typically treats each of the above tasks in isolation. In this paper, we apply a hierarchical parameter sharing technique using Conditional Random Fields for fine-grained opinion analysis, jointly detecting the boundaries of opinion expressions as well as determining two of their key attributes polarity and intensity. Our experimental results show that our proposed approach improves the performance over a baseline that does not — exploit hierarchical structure among the classes. In addition, we find that the joint approach outperforms a baseline that is based on cascading two separate components.
4 0.83342075 22 acl-2010-A Unified Graph Model for Sentence-Based Opinion Retrieval
Author: Binyang Li ; Lanjun Zhou ; Shi Feng ; Kam-Fai Wong
Abstract: There is a growing research interest in opinion retrieval as on-line users’ opinions are becoming more and more popular in business, social networks, etc. Practically speaking, the goal of opinion retrieval is to retrieve documents, which entail opinions or comments, relevant to a target subject specified by the user’s query. A fundamental challenge in opinion retrieval is information representation. Existing research focuses on document-based approaches and documents are represented by bag-of-word. However, due to loss of contextual information, this representation fails to capture the associative information between an opinion and its corresponding target. It cannot distinguish different degrees of a sentiment word when associated with different targets. This in turn seriously affects opinion retrieval performance. In this paper, we propose a sentence-based approach based on a new information representa- , tion, namely topic-sentiment word pair, to capture intra-sentence contextual information between an opinion and its target. Additionally, we consider inter-sentence information to capture the relationships among the opinions on the same topic. Finally, the two types of information are combined in a unified graph-based model, which can effectively rank the documents. Compared with existing approaches, experimental results on the COAE08 dataset showed that our graph-based model achieved significant improvement. 1
5 0.6971283 123 acl-2010-Generating Focused Topic-Specific Sentiment Lexicons
Author: Valentin Jijkoun ; Maarten de Rijke ; Wouter Weerkamp
Abstract: We present a method for automatically generating focused and accurate topicspecific subjectivity lexicons from a general purpose polarity lexicon that allow users to pin-point subjective on-topic information in a set of relevant documents. We motivate the need for such lexicons in the field of media analysis, describe a bootstrapping method for generating a topic-specific lexicon from a general purpose polarity lexicon, and evaluate the quality of the generated lexicons both manually and using a TREC Blog track test set for opinionated blog post retrieval. Although the generated lexicons can be an order of magnitude more selective than the general purpose lexicon, they maintain, or even improve, the performance of an opin- ion retrieval system.
6 0.42046961 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
7 0.39994889 42 acl-2010-Automatically Generating Annotator Rationales to Improve Sentiment Classification
8 0.38475594 18 acl-2010-A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
9 0.37112018 157 acl-2010-Last but Definitely Not Least: On the Role of the Last Sentence in Automatic Polarity-Classification
10 0.36826023 256 acl-2010-Vocabulary Choice as an Indicator of Perspective
11 0.34700361 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
12 0.32392788 141 acl-2010-Identifying Text Polarity Using Random Walks
13 0.31013229 58 acl-2010-Classification of Feedback Expressions in Multimodal Data
14 0.30183697 105 acl-2010-Evaluating Multilanguage-Comparability of Subjectivity Analysis Systems
15 0.29550919 199 acl-2010-Preferences versus Adaptation during Referring Expression Generation
16 0.29517326 4 acl-2010-A Cognitive Cost Model of Annotations Based on Eye-Tracking Data
17 0.29430601 140 acl-2010-Identifying Non-Explicit Citing Sentences for Citation-Based Summarization.
18 0.29197925 2 acl-2010-"Was It Good? It Was Provocative." Learning the Meaning of Scalar Adjectives
19 0.28536481 113 acl-2010-Extraction and Approximation of Numerical Attributes from the Web
20 0.2836844 230 acl-2010-The Manually Annotated Sub-Corpus: A Community Resource for and by the People
topicId topicWeight
[(7, 0.022), (25, 0.047), (38, 0.188), (39, 0.012), (42, 0.126), (44, 0.01), (59, 0.059), (60, 0.01), (72, 0.043), (73, 0.065), (76, 0.017), (78, 0.027), (80, 0.026), (83, 0.124), (84, 0.028), (98, 0.099)]
simIndex simValue paperId paperTitle
same-paper 1 0.83871508 208 acl-2010-Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
Author: Cigdem Toprak ; Niklas Jakob ; Iryna Gurevych
Abstract: In this paper, we introduce a corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information. We present a two-level annotation scheme. In the first stage, the reviews are analyzed at the sentence level for (i) relevancy to a given topic, and (ii) expressing an evaluation about the topic. In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, intensity), and the functional components of the evaluations (opinion terms, targets and holders). We discuss the annotation scheme, the inter-annotator agreement for different subtasks and our observations.
2 0.73351514 149 acl-2010-Incorporating Extra-Linguistic Information into Reference Resolution in Collaborative Task Dialogue
Author: Ryu Iida ; Syumpei Kobayashi ; Takenobu Tokunaga
Abstract: This paper proposes an approach to reference resolution in situated dialogues by exploiting extra-linguistic information. Recently, investigations of referential behaviours involved in situations in the real world have received increasing attention by researchers (Di Eugenio et al., 2000; Byron, 2005; van Deemter, 2007; Spanger et al., 2009). In order to create an accurate reference resolution model, we need to handle extra-linguistic information as well as textual information examined by existing approaches (Soon et al., 2001 ; Ng and Cardie, 2002, etc.). In this paper, we incorporate extra-linguistic information into an existing corpus-based reference resolution model, and investigate its effects on refer- ence resolution problems within a corpus of Japanese dialogues. The results demonstrate that our proposed model achieves an accuracy of 79.0% for this task.
3 0.72517037 251 acl-2010-Using Anaphora Resolution to Improve Opinion Target Identification in Movie Reviews
Author: Niklas Jakob ; Iryna Gurevych
Abstract: unkown-abstract
4 0.72423255 123 acl-2010-Generating Focused Topic-Specific Sentiment Lexicons
Author: Valentin Jijkoun ; Maarten de Rijke ; Wouter Weerkamp
Abstract: We present a method for automatically generating focused and accurate topicspecific subjectivity lexicons from a general purpose polarity lexicon that allow users to pin-point subjective on-topic information in a set of relevant documents. We motivate the need for such lexicons in the field of media analysis, describe a bootstrapping method for generating a topic-specific lexicon from a general purpose polarity lexicon, and evaluate the quality of the generated lexicons both manually and using a TREC Blog track test set for opinionated blog post retrieval. Although the generated lexicons can be an order of magnitude more selective than the general purpose lexicon, they maintain, or even improve, the performance of an opin- ion retrieval system.
5 0.7111662 178 acl-2010-Non-Cooperation in Dialogue
Author: Brian Pluss
Abstract: This paper presents ongoing research on computational models for non-cooperative dialogue. We start by analysing different levels of cooperation in conversation. Then, inspired by findings from an empirical study, we propose a technique for measuring non-cooperation in political interviews. Finally, we describe a research programme towards obtaining a suitable model and discuss previous accounts for conflictive dialogue, identifying the differences with our work.
6 0.70803583 134 acl-2010-Hierarchical Sequential Learning for Extracting Opinions and Their Attributes
7 0.70417875 150 acl-2010-Inducing Domain-Specific Semantic Class Taggers from (Almost) Nothing
8 0.6879614 214 acl-2010-Sparsity in Dependency Grammar Induction
9 0.68325716 231 acl-2010-The Prevalence of Descriptive Referring Expressions in News and Narrative
10 0.68006796 22 acl-2010-A Unified Graph Model for Sentence-Based Opinion Retrieval
11 0.66635311 33 acl-2010-Assessing the Role of Discourse References in Entailment Inference
12 0.6619544 1 acl-2010-"Ask Not What Textual Entailment Can Do for You..."
13 0.65632463 73 acl-2010-Coreference Resolution with Reconcile
14 0.65608817 112 acl-2010-Extracting Social Networks from Literary Fiction
15 0.65247375 42 acl-2010-Automatically Generating Annotator Rationales to Improve Sentiment Classification
16 0.64820957 219 acl-2010-Supervised Noun Phrase Coreference Research: The First Fifteen Years
17 0.64425397 122 acl-2010-Generating Fine-Grained Reviews of Songs from Album Reviews
18 0.64188832 101 acl-2010-Entity-Based Local Coherence Modelling Using Topological Fields
19 0.64073539 209 acl-2010-Sentiment Learning on Product Reviews via Sentiment Ontology Tree
20 0.63898993 174 acl-2010-Modeling Semantic Relevance for Question-Answer Pairs in Web Social Communities