emnlp emnlp2011 emnlp2011-120 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Richard Socher ; Jeffrey Pennington ; Eric H. Huang ; Andrew Y. Ng ; Christopher D. Manning
Abstract: We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. [sent-8, score-0.942]
2 In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. [sent-10, score-1.331]
3 We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. [sent-11, score-0.769]
4 The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. [sent-12, score-0.276]
5 Detecting sentiment in these data is a challenging task which has recently spawned a lot of interest (Pang and Lee, 2008). [sent-16, score-0.445]
6 Current baseline methods often use bag-of-words representations which cannot properly capture more complex linguistic phenomena in sentiment analysis (Pang et al. [sent-17, score-0.578]
7 , 151 Recursive Autoencoder Soiry,HuwgsalkYeoudR ckintToe h aIUndersptandkeWow,cJausrtWowISRDPWneirdmseoptrcdi aesbmcnteu nidctoanis Figure 1: Illustration of our recursive autoencoder architecture which learns semantic vector representations of phrases. [sent-21, score-0.596]
8 Then they are recursively merged by the same autoencoder network into a fixed length sentence representation. [sent-23, score-0.231]
9 The vectors at each node are used as features to predict a distribution over sentiment labels. [sent-24, score-0.666]
10 (ii) Our system can be trained both on unlabeled domain data and on supervised sentiment data and does not require any language-specific sentiment lexica, Proce Ed iningbsu orfg th ,e S 2c0o1tl1an Cdo,n UfeKr,en Jcuely on 27 E–m31p,ir 2ic0a1l1 M. [sent-34, score-0.89]
11 (iii) Rather than limiting sentiment to a positive/negative scale, we predict a multidimensional distribution over several complex, interconnected sentiments. [sent-37, score-0.574]
12 We introduce an approach based on semisupervised, recursive autoencoders (RAE) which use as input continuous word vectors. [sent-38, score-0.446]
13 1 shows an illustration of the model which learns vector representations of phrases and full sentences as well as their hierarchical structure from unsupervised text. [sent-40, score-0.208]
14 We extend our model to also learn a distribution over sentiment labels at each node of the hierarchy. [sent-41, score-0.525]
15 The dataset consists of very personal confessions anonymously made by people on the experience project website www. [sent-44, score-0.299]
16 Reaction labels are you rock (expressing approvement), tehee (amusement), I understand, Sorry, hugs and Wow, just wow (displaying shock). [sent-48, score-0.182]
17 For evaluation on this dataset we predict both the label with the most votes as well as the full distribution over the sentiment categories. [sent-49, score-0.739]
18 We first describe neural word representations and then proceed to review a related recursive model based on autoencoders, introduce our recursive autoencoder (RAE) and describe how it can be modified to jointly 152 learn phrase representations, phrase structure and sentiment distributions. [sent-57, score-1.226]
19 T ∈h iRs initialization |wVo |rk iss twheell s zine supervised settings Twhihser ine a anleiztwatoiornk can subsequently modify these vectors to capture certain label distributions. [sent-63, score-0.178]
20 These models jointly learn an embedding of words into a vector space and use these vectors to predict how likely a word occurs given its context. [sent-66, score-0.213]
21 Figure 2: Illustration of an application of a recursive autoencoder to a binary tree. [sent-78, score-0.428]
22 The nodes which are not filled are only used to compute reconstruction errors. [sent-79, score-0.273]
23 A standard autoencoder (in box) is re-used at each node of the tree. [sent-80, score-0.273]
24 2 Traditional Recursive Autoencoders The goal of autoencoders is to learn a representation of their inputs. [sent-82, score-0.249]
25 In the past autoencoders have only been used in setting where the tree structure was given a-priori. [sent-84, score-0.302]
26 2 shows an instance of a recursive autoencoder (RAE) applied to a given tree. [sent-87, score-0.428]
27 The first parent vector y1 is computed from the children (c1, c2) = (x3, x4) : p = f(W(1)[c1; c2] + b(1)), (2) W(1) where we multiplied a matrix of parameters ∈ Rn×2n by the concatenation of the two children∈. [sent-97, score-0.194]
28 One way of assessing how well this ndimensional vector represents its children is to try to reconstruct the children in a reconstruction layer: ? [sent-99, score-0.48]
29 the goal is to minimize the reconstruction errors of this input pair. [sent-104, score-0.273]
30 Now that we have defined how an autoencoder can be used to compute an n-dimensional vector representation (p) of two n-dimensional children (c1, c2), we can describe how such a network can be used for the rest of the tree. [sent-128, score-0.354]
31 Again, after computing the intermediate parent vector y2, we can assess how well this vector capture the content of the children by computing the reconstruction error as in Eq. [sent-132, score-0.598]
32 The process repeat until the full tree is constructed and we have a reconstruction error at each nonterminal node. [sent-134, score-0.418]
33 The goal of our structureprediction RAE is to minimize the reconstruction error of all vector pairs of children in a tree. [sent-138, score-0.453]
34 For a sentence with m words, we apply the autoencoder recursively. [sent-144, score-0.231]
35 For each word pair, we save the potential parent node p and the resulting reconstruction error. [sent-146, score-0.386]
36 After computing the score for the first pair, the network is shifted by one position and takes as input vectors (c1, c2) = (x2, x3) and again computes a potential parent node and a score. [sent-147, score-0.205]
37 Next, it selects the pair which had the lowest reconstruction error (Erec) and its parent representation p will represent this phrase and replace both children in the sentence word list. [sent-149, score-0.485]
38 One problem with simply using the reconstruction error of both children equally as describe in Eq. [sent-162, score-0.414]
39 4 is that each child could represent a different number of previously collapsed words and is hence of bigger importance for the overall meaning reconstruction of the sentence. [sent-163, score-0.273]
40 We capture this desideratum by adjusting the reconstruction error. [sent-165, score-0.308]
41 Let n1, n2 be 154 the number of words underneath a current potential child, we re-define the reconstruction error to be = n1n+1 n2? [sent-166, score-0.361]
42 The RAE tries to lower reconstruction error of not only the bigrams but also of nodes higher in the tree. [sent-200, score-0.33]
43 Unfortunately, since the RAE computes the hidden representations it then tries to reconstruct, it can just lower reconstruction error by making the hidden layer very small in magnitude. [sent-201, score-0.515]
44 1 One of the main advantages of the RAE is that each node of the tree built by the RAE has associated with it a distributed vector representation (the parent vector p) which could also be seen as features describing that phrase. [sent-207, score-0.244]
45 We can leverage this representation by adding on top of each parent node a simple softmax layer to predict class distributions: d(p; θ) = softmax(Wlabelp). [sent-208, score-0.364]
46 Reconstruction error Cross-entropy error W(2)W(label) W(1) Figure 3: Illustration of an RAE unit at a nonterminal tree node. [sent-215, score-0.202]
47 Red nodes show the supervised softmax layer for label distribution prediction. [sent-216, score-0.259]
48 Using this cross-entropy error for the label and the reconstruction error from Eq. [sent-217, score-0.438]
49 The error at each nonterminal node is the weighted sum of reconstruction and cross-entropy errors, E( [c1; c2]s ps t, θ) ,, = αErec( [c1; c2]s ; θ) + (1 − α)EcE (ps, t; θ) . [sent-219, score-0.407]
50 The hyperparameter α weighs reconstruction and cross-entropy error. [sent-220, score-0.273]
51 When minimizing the crossentropy error of this softmax layer, the error will backpropagate and influence both the RAE parameters and the word representations. [sent-221, score-0.197]
52 When learning with positive/negative sentiment, the word embeddings get modified and capture less syntactic and more sentiment information. [sent-224, score-0.48]
53 In order to predict the sentiment distribution of a sentence with this model, we use the learned vector representation of the top tree node and train a simple logistic regression classifier. [sent-225, score-0.666]
54 W(1), 4 Experiments We first describe the new experience project (EP) dataset, results of standard classification tasks on this dataset and how to predict its sentiment label distributions. [sent-230, score-0.66]
55 Note that alternatives such as Brown clusters are not suitable since they do not capture sentiment information (good and bad are usually in the same cluster) and cannot be modified via backpropagation. [sent-240, score-0.48]
56 is the distribution of the different classes (in the case of 2, the positive/negative classes, for EP the rounded distribution of total votes in each class). [sent-260, score-0.184]
57 1 EP Dataset: The Experience Project The confessions section of the experience project website3 lets people anonymously write short personal stories or “confessions”. [sent-264, score-0.302]
58 The EP dataset has 3 1,676 confession entries, a to- tal number of 74,859 votes for the 5 labels above, the average number of votes per entry is 2. [sent-274, score-0.264]
59 Table 1 shows statistics of this and other commonly used sentiment datasets (which we compare on in later experiments). [sent-288, score-0.482]
60 Baseline 2: Features This model is similar to traditional approaches to sentiment classification in that it uses many hand-engineered resources. [sent-306, score-0.445]
61 We then replaced sentiment words with a sentiment category identifier using the sentiment lexica of the Harvard Inquirer (Stone, 1966) and LIWC (Pennebaker et al. [sent-308, score-1.488]
62 4 Binary Polarity Classification In order to compare our approach to other methods we also show results on commonly used sentiment datasets: movie reviews4 (MR) (Pang and Lee, 2005) and opinions5 (MPQA) (Wiebe et al. [sent-349, score-0.528]
63 We use the same training and testing regimen (10-fold cross validation) as well as their baselines: majority phrase voting using sentiment and reversal lexica; rule-based reversal using a dependency tree; Bag-of-Features and their full Tree-CRF model. [sent-354, score-0.555]
64 Correctly classifying these instances can only be the result of having them in the original sentiment lexicon. [sent-358, score-0.445]
65 Hence, for the experiment on MPQA we added the same sentiment lexicon that (Nakagawa et al. [sent-359, score-0.445]
66 4 Table 4: Accuracy of sentiment classification on movie review polarity (MR) and the MPQA dataset. [sent-383, score-0.635]
67 t8oftheM1R polarity dataset for different weightings of reconstruction error and supervised cross-entropy error: err = αErec + (1 α)EcE. [sent-389, score-0.485]
68 We visualize the semantic vectors that the recursive autoencoder learns by listing n-grams that give the highest probability for each polarity. [sent-392, score-0.551]
69 Table 5 shows such n-grams for different lengths when the RAE is trained on the movie review polarity dataset. [sent-393, score-0.19]
70 2 for the reconstruction error prevents overfitting and achieves the highest performance. [sent-402, score-0.33]
71 1 Autoencoders and Deep Learning Autoencoders are neural networks that learn a reduced dimensional representation of fixed-size inputs such as image patches or bag-of-word representations of text documents. [sent-405, score-0.216]
72 (2010) learn dynamic autoencoders for documents in a bag- of-words format which, like ours, combine supervised and reconstruction objectives. [sent-408, score-0.522]
73 The idea of applying an autoencoder in a recursive setting was introduced by Pollack (1990). [sent-409, score-0.428]
74 Pollack’s recursive auto-associative memories (RAAMs) are similar to ours in that they are a connectionst, feedforward model. [sent-410, score-0.197]
75 However, RAAMs learn vector representations only for fixed recursive data structures, whereas our RAE builds this recursive data structure. [sent-411, score-0.531]
76 One of the major shortcomings of previous applications of recursive autoencoders to natural language sentences was their binary word representation as discussed in Sec. [sent-413, score-0.446]
77 , 2011) introduced a max-margin framework based on recursive neural networks (RNNs) for labeled structure prediction. [sent-418, score-0.315]
78 The current work is related in that it uses a recursive deep learning model. [sent-420, score-0.256]
79 Other recent deep learning methods for sentiment analysis include (Maas et al. [sent-424, score-0.504]
80 (2002) were one of the first to experiment with sentiment classification. [sent-428, score-0.445]
81 They show that simple bag-of-words approaches based on Naive Bayes, MaxEnt models or SVMs are often insufficient for predicting sentiment of documents even though they work well for general topic-based document classification. [sent-429, score-0.48]
82 Other document-level sentiment work includes (Turney, 2002; Dave et al. [sent-431, score-0.445]
83 Instead of document level sentiment classification, (Wilson et al. [sent-435, score-0.445]
84 Our model naturally incorporates the recursive interaction between context and polarity words in sentences in a unified framework while simultaneously learning the necessary features to make accurate predictions. [sent-438, score-0.304]
85 Other approaches for sentence-level sentiment detection include (Yu and Hatzivassiloglou, 2003; Grefenstette et al. [sent-439, score-0.445]
86 Most previous work is centered around a given sentiment lexicon or building one via heuristics (Kim and Hovy, 2007; Esuli and Sebastiani, 2007), manual annotation (Das and Chen, 2001) or machine learning techniques (Turney, 2002). [sent-442, score-0.445]
87 In contrast, we do not require an initial or constructed sentiment lexicon of positive and negative words. [sent-443, score-0.445]
88 In fact, when training our approach on documents or sentences, it jointly learns such lexica for both single words and n-grams (see Table 5). [sent-444, score-0.184]
89 The work of (Polanyi and Zaenen, 2006; Choi and Cardie, 2008) focuses on manually constructing several lexica and rules for both polar words and related content-word negators, such as “prevent cancer”, where prevent reverses the negative polarity of cancer. [sent-446, score-0.302]
90 , 2010) showed how to use a seed lexicon and a graph propagation framework to learn a larger sentiment lexicon that also includes polar multi-word phrases such as “once in a life time”. [sent-450, score-0.487]
91 We outperform them on the standard corpora that we tested on without requiring external systems such as POS taggers, dependency parsers and sentiment lexica. [sent-455, score-0.445]
92 6 Conclusion We presented a novel algorithm that can accurately predict sentence-level sentiment distributions. [sent-460, score-0.526]
93 Without using any hand-engineered resources such as sentiment lexica, parsers or sentiment shifting rules, our model achieves state-of-the-art performance on commonly used sentiment datasets. [sent-461, score-1.335]
94 Learning with compositional semantics as structural inference for subsentential sentiment analysis. [sent-501, score-0.445]
95 for Amazon: Extracting market sentiment from stock message boards. [sent-514, score-0.445]
96 Learning to shift the polarity of words for sentiment classification. [sent-566, score-0.552]
97 Dependency tree-based sentiment classification using CRFs with hidden variables. [sent-606, score-0.445]
98 Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. [sent-618, score-0.477]
99 Learning continuous phrase representations and syntactic parsing with recursive neural networks. [sent-674, score-0.353]
100 Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. [sent-738, score-0.237]
wordName wordTfidf (topN-words)
[('sentiment', 0.445), ('rae', 0.355), ('reconstruction', 0.273), ('autoencoders', 0.249), ('autoencoder', 0.231), ('recursive', 0.197), ('lexica', 0.153), ('ep', 0.152), ('erec', 0.124), ('socher', 0.124), ('votes', 0.108), ('polarity', 0.107), ('confessions', 0.107), ('representations', 0.098), ('pang', 0.095), ('vectors', 0.092), ('layer', 0.087), ('children', 0.084), ('movie', 0.083), ('softmax', 0.083), ('wow', 0.083), ('nakagawa', 0.077), ('mpqa', 0.076), ('pollack', 0.076), ('parent', 0.071), ('experience', 0.067), ('reaction', 0.064), ('networks', 0.06), ('kl', 0.06), ('deep', 0.059), ('neural', 0.058), ('mr', 0.057), ('error', 0.057), ('reversal', 0.055), ('hugs', 0.053), ('maas', 0.053), ('raams', 0.053), ('raes', 0.053), ('sorry', 0.053), ('voegtlin', 0.053), ('tree', 0.053), ('distributions', 0.053), ('label', 0.051), ('stories', 0.051), ('predict', 0.049), ('dataset', 0.048), ('opinion', 0.047), ('triplets', 0.046), ('rock', 0.046), ('xm', 0.046), ('divergence', 0.045), ('node', 0.042), ('stars', 0.042), ('collobert', 0.042), ('polar', 0.042), ('interconnected', 0.042), ('personal', 0.041), ('illustration', 0.04), ('vector', 0.039), ('distribution', 0.038), ('understand', 0.037), ('datasets', 0.037), ('opinions', 0.036), ('entries', 0.036), ('wiebe', 0.036), ('multinomial', 0.036), ('anonymously', 0.036), ('backpropagation', 0.036), ('beineke', 0.036), ('destroying', 0.036), ('dominey', 0.036), ('ece', 0.036), ('goller', 0.036), ('isotonic', 0.036), ('mirowski', 0.036), ('potts', 0.036), ('raam', 0.036), ('rnns', 0.036), ('teehee', 0.036), ('capture', 0.035), ('predicting', 0.035), ('greedy', 0.035), ('nonterminal', 0.035), ('emotions', 0.034), ('reviews', 0.034), ('embedding', 0.033), ('class', 0.032), ('accurately', 0.032), ('user', 0.031), ('learns', 0.031), ('thumbs', 0.031), ('wilson', 0.031), ('predicted', 0.031), ('weston', 0.031), ('underneath', 0.031), ('ikeda', 0.031), ('pennebaker', 0.031), ('lastly', 0.031), ('emotional', 0.031), ('horrible', 0.031)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000004 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Author: Richard Socher ; Jeffrey Pennington ; Eric H. Huang ; Andrew Y. Ng ; Christopher D. Manning
Abstract: We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.
2 0.32339135 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
Author: Ainur Yessenalina ; Claire Cardie
Abstract: We present a general learning-based approach for phrase-level sentiment analysis that adopts an ordinal sentiment scale and is explicitly compositional in nature. Thus, we can model the compositional effects required for accurate assignment of phrase-level sentiment. For example, combining an adverb (e.g., “very”) with a positive polar adjective (e.g., “good”) produces a phrase (“very good”) with increased polarity over the adjective alone. Inspired by recent work on distributional approaches to compositionality, we model each word as a matrix and combine words using iterated matrix multiplication, which allows for the modeling of both additive and multiplicative semantic effects. Although the multiplication-based matrix-space framework has been shown to be a theoretically elegant way to model composition (Rudolph and Giesbrecht, 2010), training such models has to be done carefully: the optimization is nonconvex and requires a good initial starting point. This paper presents the first such algorithm for learning a matrix-space model for semantic composition. In the context of the phrase-level sentiment analysis task, our experimental results show statistically significant improvements in performance over a bagof-words model.
3 0.23193276 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
Author: Samuel Brody ; Nicholas Diakopoulos
Abstract: We present an automatic method which leverages word lengthening to adapt a sentiment lexicon specifically for Twitter and similar social messaging networks. The contributions of the paper are as follows. First, we call attention to lengthening as a widespread phenomenon in microblogs and social messaging, and demonstrate the importance of handling it correctly. We then show that lengthening is strongly associated with subjectivity and sentiment. Finally, we present an automatic method which leverages this association to detect domain-specific sentiment- and emotionbearing words. We evaluate our method by comparison to human judgments, and analyze its strengths and weaknesses. Our results are of interest to anyone analyzing sentiment in microblogs and social networks, whether for research or commercial purposes.
4 0.23001069 63 emnlp-2011-Harnessing WordNet Senses for Supervised Sentiment Classification
Author: Balamurali AR ; Aditya Joshi ; Pushpak Bhattacharyya
Abstract: Traditional approaches to sentiment classification rely on lexical features, syntax-based features or a combination of the two. We propose semantic features using word senses for a supervised document-level sentiment classifier. To highlight the benefit of sense-based features, we compare word-based representation of documents with a sense-based representation where WordNet senses of the words are used as features. In addition, we highlight the benefit of senses by presenting a part-ofspeech-wise effect on sentiment classification. Finally, we show that even if a WSD engine disambiguates between a limited set of words in a document, a sentiment classifier still performs better than what it does in absence of sense annotation. Since word senses used as features show promise, we also examine the possibility of using similarity metrics defined on WordNet to address the problem of not finding a sense in the training corpus. We per- form experiments using three popular similarity metrics to mitigate the effect of unknown synsets in a test corpus by replacing them with similar synsets from the training corpus. The results show promising improvement with respect to the baseline.
5 0.16184449 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
Author: Yuanbin Wu ; Qi Zhang ; Xuanjing Huang ; Lide Wu
Abstract: Based on analysis of on-line review corpus we observe that most sentences have complicated opinion structures and they cannot be well represented by existing methods, such as frame-based and feature-based ones. In this work, we propose a novel graph-based representation for sentence level sentiment. An integer linear programming-based structural learning method is then introduced to produce the graph representations of input sentences. Experimental evaluations on a manually labeled Chinese corpus demonstrate the effectiveness of the proposed approach.
6 0.15638943 17 emnlp-2011-Active Learning with Amazon Mechanical Turk
7 0.14764719 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
9 0.090810485 142 emnlp-2011-Unsupervised Discovery of Discourse Relations for Eliminating Intra-sentence Polarity Ambiguities
10 0.076845676 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
11 0.076480128 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
12 0.073049642 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
13 0.065309212 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
14 0.054735098 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
15 0.052208569 50 emnlp-2011-Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation
16 0.052123062 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
17 0.050960943 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
18 0.050696157 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
19 0.050494183 77 emnlp-2011-Large-Scale Cognate Recovery
20 0.045527715 61 emnlp-2011-Generating Aspect-oriented Multi-Document Summarization with Event-aspect model
topicId topicWeight
[(0, 0.214), (1, -0.213), (2, 0.123), (3, 0.105), (4, 0.498), (5, 0.05), (6, 0.075), (7, 0.029), (8, 0.024), (9, 0.065), (10, 0.042), (11, 0.134), (12, -0.011), (13, 0.069), (14, 0.054), (15, -0.009), (16, 0.016), (17, -0.006), (18, -0.021), (19, 0.029), (20, 0.006), (21, 0.021), (22, -0.047), (23, -0.068), (24, 0.0), (25, 0.032), (26, 0.048), (27, -0.01), (28, 0.0), (29, -0.011), (30, 0.001), (31, -0.1), (32, 0.032), (33, -0.069), (34, -0.011), (35, 0.008), (36, 0.013), (37, 0.017), (38, 0.015), (39, 0.033), (40, 0.001), (41, -0.045), (42, 0.102), (43, 0.018), (44, 0.023), (45, 0.039), (46, -0.008), (47, -0.014), (48, 0.061), (49, 0.004)]
simIndex simValue paperId paperTitle
same-paper 1 0.95126599 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Author: Richard Socher ; Jeffrey Pennington ; Eric H. Huang ; Andrew Y. Ng ; Christopher D. Manning
Abstract: We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.
2 0.91173249 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
Author: Ainur Yessenalina ; Claire Cardie
Abstract: We present a general learning-based approach for phrase-level sentiment analysis that adopts an ordinal sentiment scale and is explicitly compositional in nature. Thus, we can model the compositional effects required for accurate assignment of phrase-level sentiment. For example, combining an adverb (e.g., “very”) with a positive polar adjective (e.g., “good”) produces a phrase (“very good”) with increased polarity over the adjective alone. Inspired by recent work on distributional approaches to compositionality, we model each word as a matrix and combine words using iterated matrix multiplication, which allows for the modeling of both additive and multiplicative semantic effects. Although the multiplication-based matrix-space framework has been shown to be a theoretically elegant way to model composition (Rudolph and Giesbrecht, 2010), training such models has to be done carefully: the optimization is nonconvex and requires a good initial starting point. This paper presents the first such algorithm for learning a matrix-space model for semantic composition. In the context of the phrase-level sentiment analysis task, our experimental results show statistically significant improvements in performance over a bagof-words model.
3 0.84783465 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
Author: Samuel Brody ; Nicholas Diakopoulos
Abstract: We present an automatic method which leverages word lengthening to adapt a sentiment lexicon specifically for Twitter and similar social messaging networks. The contributions of the paper are as follows. First, we call attention to lengthening as a widespread phenomenon in microblogs and social messaging, and demonstrate the importance of handling it correctly. We then show that lengthening is strongly associated with subjectivity and sentiment. Finally, we present an automatic method which leverages this association to detect domain-specific sentiment- and emotionbearing words. We evaluate our method by comparison to human judgments, and analyze its strengths and weaknesses. Our results are of interest to anyone analyzing sentiment in microblogs and social networks, whether for research or commercial purposes.
4 0.71222681 63 emnlp-2011-Harnessing WordNet Senses for Supervised Sentiment Classification
Author: Balamurali AR ; Aditya Joshi ; Pushpak Bhattacharyya
Abstract: Traditional approaches to sentiment classification rely on lexical features, syntax-based features or a combination of the two. We propose semantic features using word senses for a supervised document-level sentiment classifier. To highlight the benefit of sense-based features, we compare word-based representation of documents with a sense-based representation where WordNet senses of the words are used as features. In addition, we highlight the benefit of senses by presenting a part-ofspeech-wise effect on sentiment classification. Finally, we show that even if a WSD engine disambiguates between a limited set of words in a document, a sentiment classifier still performs better than what it does in absence of sense annotation. Since word senses used as features show promise, we also examine the possibility of using similarity metrics defined on WordNet to address the problem of not finding a sense in the training corpus. We per- form experiments using three popular similarity metrics to mitigate the effect of unknown synsets in a test corpus by replacing them with similar synsets from the training corpus. The results show promising improvement with respect to the baseline.
5 0.66657901 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
Author: Song Feng ; Ritwik Bose ; Yejin Choi
Abstract: In this paper, we introduce a connotation lexicon, a new type of lexicon that lists words with connotative polarity, i.e., words with positive connotation (e.g., award, promotion) and words with negative connotation (e.g., cancer, war). Connotation lexicons differ from much studied sentiment lexicons: the latter concerns words that express sentiment, while the former concerns words that evoke or associate with a specific polarity of sentiment. Understanding the connotation of words would seem to require common sense and world knowledge. However, we demonstrate that much of the connotative polarity of words can be inferred from natural language text in a nearly unsupervised manner. The key linguistic insight behind our approach is selectional preference of connotative predicates. We present graphbased algorithms using PageRank and HITS that collectively learn connotation lexicon together with connotative predicates. Our empirical study demonstrates that the resulting connotation lexicon is of great value for sentiment analysis complementing existing sentiment lexicons.
6 0.57368279 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
7 0.48144531 17 emnlp-2011-Active Learning with Amazon Mechanical Turk
9 0.29678431 142 emnlp-2011-Unsupervised Discovery of Discourse Relations for Eliminating Intra-sentence Polarity Ambiguities
10 0.29517508 28 emnlp-2011-Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances
11 0.2950618 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
12 0.26841879 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
13 0.25680363 71 emnlp-2011-Identifying and Following Expert Investors in Stock Microblogs
14 0.24381448 46 emnlp-2011-Efficient Subsampling for Training Complex Language Models
15 0.24149361 111 emnlp-2011-Reducing Grounded Learning Tasks To Grammatical Inference
16 0.22805622 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
17 0.22478418 82 emnlp-2011-Learning Local Content Shift Detectors from Document-level Information
18 0.21309423 10 emnlp-2011-A Probabilistic Forest-to-String Model for Language Generation from Typed Lambda Calculus Expressions
19 0.20861211 131 emnlp-2011-Syntactic Decision Tree LMs: Random Selection or Intelligent Design?
20 0.20179592 47 emnlp-2011-Efficient retrieval of tree translation examples for Syntax-Based Machine Translation
topicId topicWeight
[(15, 0.344), (23, 0.097), (36, 0.022), (37, 0.037), (45, 0.07), (53, 0.011), (54, 0.035), (57, 0.02), (62, 0.025), (64, 0.022), (66, 0.033), (69, 0.014), (79, 0.05), (82, 0.016), (90, 0.016), (96, 0.032), (97, 0.067), (98, 0.03)]
simIndex simValue paperId paperTitle
1 0.77251923 99 emnlp-2011-Non-parametric Bayesian Segmentation of Japanese Noun Phrases
Author: Yugo Murawaki ; Sadao Kurohashi
Abstract: A key factor of high quality word segmentation for Japanese is a high-coverage dictionary, but it is costly to manually build such a lexical resource. Although external lexical resources for human readers are potentially good knowledge sources, they have not been utilized due to differences in segmentation criteria. To supplement a morphological dictionary with these resources, we propose a new task of Japanese noun phrase segmentation. We apply non-parametric Bayesian language models to segment each noun phrase in these resources according to the statistical behavior of its supposed constituents in text. For inference, we propose a novel block sampling procedure named hybrid type-based sampling, which has the ability to directly escape a local optimum that is not too distant from the global optimum. Experiments show that the proposed method efficiently corrects the initial segmentation given by a morphological ana- lyzer.
same-paper 2 0.76211894 120 emnlp-2011-Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Author: Richard Socher ; Jeffrey Pennington ; Eric H. Huang ; Andrew Y. Ng ; Christopher D. Manning
Abstract: We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model’s ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.
3 0.58015299 8 emnlp-2011-A Model of Discourse Predictions in Human Sentence Processing
Author: Amit Dubey ; Frank Keller ; Patrick Sturt
Abstract: This paper introduces a psycholinguistic model of sentence processing which combines a Hidden Markov Model noun phrase chunker with a co-reference classifier. Both models are fully incremental and generative, giving probabilities of lexical elements conditional upon linguistic structure. This allows us to compute the information theoretic measure of surprisal, which is known to correlate with human processing effort. We evaluate our surprisal predictions on the Dundee corpus of eye-movement data show that our model achieve a better fit with human reading times than a syntax-only model which does not have access to co-reference information.
4 0.46380386 30 emnlp-2011-Compositional Matrix-Space Models for Sentiment Analysis
Author: Ainur Yessenalina ; Claire Cardie
Abstract: We present a general learning-based approach for phrase-level sentiment analysis that adopts an ordinal sentiment scale and is explicitly compositional in nature. Thus, we can model the compositional effects required for accurate assignment of phrase-level sentiment. For example, combining an adverb (e.g., “very”) with a positive polar adjective (e.g., “good”) produces a phrase (“very good”) with increased polarity over the adjective alone. Inspired by recent work on distributional approaches to compositionality, we model each word as a matrix and combine words using iterated matrix multiplication, which allows for the modeling of both additive and multiplicative semantic effects. Although the multiplication-based matrix-space framework has been shown to be a theoretically elegant way to model composition (Rudolph and Giesbrecht, 2010), training such models has to be done carefully: the optimization is nonconvex and requires a good initial starting point. This paper presents the first such algorithm for learning a matrix-space model for semantic composition. In the context of the phrase-level sentiment analysis task, our experimental results show statistically significant improvements in performance over a bagof-words model.
5 0.45388776 104 emnlp-2011-Personalized Recommendation of User Comments via Factor Models
Author: Deepak Agarwal ; Bee-Chung Chen ; Bo Pang
Abstract: In recent years, the amount of user-generated opinionated texts (e.g., reviews, user comments) continues to grow at a rapid speed: featured news stories on a major event easily attract thousands of user comments on a popular online News service. How to consume subjective information ofthis volume becomes an interesting and important research question. In contrast to previous work on review analysis that tried to filter or summarize information for a generic average user, we explore a different direction of enabling personalized recommendation of such information. For each user, our task is to rank the comments associated with a given article according to personalized user preference (i.e., whether the user is likely to like or dislike the comment). To this end, we propose a factor model that incorporates rater-comment and rater-author interactions simultaneously in a principled way. Our full model significantly outperforms strong baselines as well as related models that have been considered in previous work.
6 0.43081194 39 emnlp-2011-Discovering Morphological Paradigms from Plain Text Using a Dirichlet Process Mixture Model
8 0.42023394 1 emnlp-2011-A Bayesian Mixture Model for PoS Induction Using Multiple Features
9 0.41734114 126 emnlp-2011-Structural Opinion Mining for Graph-based Sentiment Representation
10 0.40844598 85 emnlp-2011-Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming
11 0.4024379 63 emnlp-2011-Harnessing WordNet Senses for Supervised Sentiment Classification
12 0.40046513 97 emnlp-2011-Multiword Expression Identification with Tree Substitution Grammars: A Parsing tour de force with French
13 0.39580232 33 emnlp-2011-Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! Using Word Lengthening to Detect Sentiment in Microblogs
14 0.3944515 81 emnlp-2011-Learning General Connotation of Words using Graph-based Algorithms
15 0.39303771 117 emnlp-2011-Rumor has it: Identifying Misinformation in Microblogs
16 0.3920483 77 emnlp-2011-Large-Scale Cognate Recovery
17 0.38896924 142 emnlp-2011-Unsupervised Discovery of Discourse Relations for Eliminating Intra-sentence Polarity Ambiguities
18 0.38772845 108 emnlp-2011-Quasi-Synchronous Phrase Dependency Grammars for Machine Translation
19 0.38586098 53 emnlp-2011-Experimental Support for a Categorical Compositional Distributional Model of Meaning
20 0.38267413 35 emnlp-2011-Correcting Semantic Collocation Errors with L1-induced Paraphrases