emnlp emnlp2013 emnlp2013-158 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Richard Socher ; Alex Perelygin ; Jean Wu ; Jason Chuang ; Christopher D. Manning ; Andrew Ng ; Christopher Potts
Abstract: Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.
Reference: text
sentIndex sentText sentNum sentScore
1 edu s Abstract Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. [sent-8, score-0.176]
2 Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. [sent-9, score-0.524]
3 It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. [sent-11, score-0.945]
4 The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80. [sent-16, score-0.501]
5 Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases. [sent-19, score-0.491]
6 Because they cannot capture the meaning of longer phrases properly, compositionality in semantic vector spaces has recently received a lot of attention (Mitchell and Lapata, 2010; Socher et al. [sent-21, score-0.394]
7 However, progress is held back by the current lack of large and labeled compositionality resources and 1631 work accurately predicting 5 sentiment classes, very negative to very positive (– 0, +, + +), at every node of a parse tree and capturing the negation and its scope in this sentence. [sent-26, score-1.041]
8 To address this need, we introduce the Stanford Sentiment Treebank and a powerful Recursive Neural Tensor Network that can accurately predict the compositional semantic effects present in this new corpus. [sent-28, score-0.249]
9 The Stanford Sentiment Treebank is the first corpus with fully labeled parse trees that allows for a complete analysis of the compositional effects of sentiment in language. [sent-29, score-0.51]
10 This new dataset allows us to analyze the intricacies of sentiment and to capture complex linguistic phenomena. [sent-32, score-0.408]
11 While there are several datasets with document and chunk labels available, there is a need to better capture sentiment from short comments, such as Twitter data, which provide less overall signal per document. [sent-38, score-0.401]
12 In order to capture the compositional effects with higher accuracy, we propose a new model called the Recursive Neural Tensor Network (RNTN). [sent-39, score-0.176]
13 They represent a phrase through word vectors and a parse tree and then compute vectors for higher nodes in the tree using the same tensor-based composition function. [sent-41, score-0.384]
14 We compare to several supervised, compositional models such as standard recursive neural networks (RNN) (Socher et al. [sent-42, score-0.493]
15 Lastly, we use a test set of positive and negative sentences and their respective negations to show that, unlike bag of words models, the RNTN accurately captures the sentiment change and scope of negation. [sent-47, score-0.698]
16 RNTNs also learn that sentiment of phrases following the contrastive conjunction ‘but’ dominates. [sent-48, score-0.466]
17 However, distributional vectors often do not properly capture the differences in antonyms since those often have similar contexts. [sent-57, score-0.173]
18 One possibility to remedy this is to use neural word vectors (Bengio et al. [sent-58, score-0.203]
19 These vectors can be trained in an unsupervised fashion to capture distributional similarities (Collobert and Weston, 2008; Huang et al. [sent-60, score-0.207]
20 , 2012) but then also be fine-tuned and trained to specific tasks such as sentiment detection (Socher et al. [sent-61, score-0.339]
21 two-word phrases and analyze similarities computed by vector addition, multiplication and others. [sent-68, score-0.183]
22 Some related models such as holographic reduced representations (Plate, 1995), quantum logic (Widdows, 2008), discrete-continuous models (Clark and Pulman, 2007) and the recent compositional matrix space model (Rudolph and Giesbrecht, 2010) have not been experimentally validated on larger corpora. [sent-69, score-0.297]
23 Yessenalina and Cardie (201 1) compute matrix representations for longer phrases and define composition as matrix multiplication, and also evaluate on sentiment. [sent-70, score-0.342]
24 In particular we will de- scribe and experimentally compare our new RNTN model to recursive neural networks (RNN) (Socher et al. [sent-73, score-0.351]
25 , 2012) both of which have been applied to bag of words sentiment corpora. [sent-75, score-0.415]
26 While these models are highly interesting and work well in closed domains and on discrete sets, they could only capture sentiment distributions using separate mechanisms beyond the currently used logical forms. [sent-78, score-0.403]
27 Apart from the above mentioned work on RNNs, several compositionality ideas related to neural networks have been discussed by Bottou (201 1) and Hinton (1990) and first models such as Recursive Auto-associative memories been experimented with by Pollack (1990). [sent-80, score-0.325]
28 The idea to relate inputs through three way interactions, parameterized by a tensor have been proposed for relation classification (Sutskever et al. [sent-81, score-0.259]
29 Apart from the abovementioned work, most approaches in sentiment analysis use bag of words representations (Pang and Lee, 2008). [sent-86, score-0.456]
30 Snyder and Barzilay (2007) analyzed larger reviews in more detail by analyzing the sentiment of multiple aspects of restaurants, such as food or atmosphere. [sent-87, score-0.339]
31 Several works have explored sentiment compositionality through careful engineering of features or polarity shifting rules on syntactic structures (Polanyi and Zaenen, 2006; Moilanen and Pulman, 2007; Rentoumi et al. [sent-88, score-0.473]
32 3 Stanford Sentiment Treebank Bag of words classifiers can work well in longer documents by relying on a few words with strong sentiment like ‘awesome’ or ‘exhilarating. [sent-91, score-0.383]
33 ’ However, sentiment accuracies even for binary positive/negative classification for single sentences has not exceeded 80% for several years. [sent-92, score-0.379]
34 In this section we will introduce and provide some analyses for the new Sentiment Treebank which includes labels for every syntactically plausible phrase in thousands of sentences, allowing us to train and evaluate compositional models. [sent-97, score-0.17]
35 Random phrases were shown and annotators had a slider for selecting the senti- ment and its degree. [sent-101, score-0.19]
36 The phrases in each hit are randomly sampled from the set of all phrases in order to prevent labels being influenced by what follows. [sent-112, score-0.224]
37 We also notice that stronger sentiment often builds up in longer phrases and the majority of the shorter phrases are neutral. [sent-118, score-0.579]
38 Another observation is that most annotators moved the slider to one of the five positions: negative, somewhat negative, neutral, positive or somewhat positive. [sent-119, score-0.274]
39 We will name this fine-grained sentiment classification and our main experiment will be to recover these five labels for phrases of all lengths. [sent-122, score-0.505]
40 (a) (b) (c)(d) Distributions of sentiment values for (a) unigrams, N-Gram Length Figure 2: Normalized histogram of sentiment annotations longer phrases are well distributed. [sent-123, score-0.82]
41 4 Recursive Neural Models The models in this section compute compositional vector representations for phrases of variable length and syntactic type. [sent-127, score-0.361]
42 When an n-gram is given to the compositional models, it is parsed into a binary tree and each leaf node, corresponding to a word, is represented as a vector. [sent-131, score-0.182]
43 Recursive neural models will then compute parent vectors in a bottom up fashion using different types of compositionality functions g. [sent-132, score-0.5]
44 We first describe the operations that the below recursive neural models have in common: word vector representations and classification. [sent-135, score-0.392]
45 Rtodr×sa| Vr |e, We can use the word vectors immediately as parameters to optimize and as feature inputs to a softmax classifier. [sent-144, score-0.178]
46 labels given the word vector via: ya = softmax(Wsa) , (1) where Ws ∈ R5×d is the sentiment classification matrix. [sent-147, score-0.457]
47 1 RNN: Recursive Neural Network The simplest member of this family of neural network models is the standard recursive neural network (Goller and K ¨uchler, 1996; Socher et al. [sent-151, score-0.511]
48 Each parent vector pi, is given to the same softmax classifier of Eq. [sent-166, score-0.202]
49 This model uses the same compositionality function as the recursive autoencoder (Socher et al. [sent-168, score-0.323]
50 2 MV-RNN: Matrix-Vector RNN The MV-RNN is linguistically motivated in that most of the parameters are associated with words and each composition function that computes vectors for longer phrases depends on the actual words being combined. [sent-173, score-0.296]
51 For the tree with (vector,matrix) nodes: 1635 (p2,P2) (b,B)(c,C) (a,A) (p1,P1) the MV-RNN computes the first parent vector and its matrix via two equations: p1= f? [sent-181, score-0.188]
52 The vectors are used for classifying each phrase using the same softmax classifier as in Eq. [sent-191, score-0.178]
53 Motivated by these ideas we ask the question: Can a single, more powerful composition function per- form better and compose aggregate meaning from smaller constituents more accurately than many input specific ones? [sent-199, score-0.17]
54 We define the output of a tensor product h ∈ Rd via the following pvuetcot ofr iaze tden nsoortat piroond aucndt hthe ∈ equivalent but more detailed notation for each slice ∈ Rd×d: V[i] h =? [sent-204, score-0.266]
55 V[1:d] where ∈ R2d×2d×d is the tensor that defines multiple bilin∈ear R Rforms. [sent-213, score-0.219]
56 The main advantage over the previous RNN model, which is a special case of the RNTN when V is set to 0, is that the tensor can directly relate input vectors. [sent-232, score-0.219]
57 Intuitively, we can interpret each slice of the tensor as capturing a specific type of composition. [sent-233, score-0.266]
58 An alternative to RNTNs would be to make the compositional function more powerful by adding a second neural network layer. [sent-234, score-0.354]
59 As mentioned above, each node has a 1636 softmax classifier trained on its vector representation to predict a given ground truth or target vector t. [sent-238, score-0.269]
60 Let δi,s ∈ Rd×1 be the softmax error vector at node i: δi,s = ? [sent-248, score-0.219]
61 showed that the recursive models worked significantly worse (over 5% drop in accuracy) when no nonlinearity was used. [sent-326, score-0.233]
62 We also compare to a model that averages neural word vectors and ignores word order (VecAvg). [sent-330, score-0.203]
63 We also analyze performance on only positive and negative sentences, ignoring the neutral class. [sent-332, score-0.322]
64 1 Fine-grained Sentiment For All Phrases The main novel experiment and evaluation metric analyze the accuracy of fine-grained sentiment classification for all phrases. [sent-335, score-0.45]
65 2 showed that a fine grained classification into 5 classes is a reasonable approximation to capture most of the data variation. [sent-337, score-0.186]
66 The recursive models work very well on shorter phrases, where negation and composition are important, while bag of features baselines perform well only with longer sentences. [sent-341, score-0.506]
67 Figure 6: Accuracy curves for fine grained sentiment classification at each n-gram lengths. [sent-393, score-0.491]
68 Hence, these experiments show the improvement even baseline methods can achieve with the sentiment treebank. [sent-398, score-0.339]
69 The combination of the new sentiment treebank and the RNTN pushes the state of the art on short phrases up to 85. [sent-403, score-0.54]
70 We analyze a strict setting, where X and Y are phrases of different sentiment (including neutral). [sent-410, score-0.472]
71 In this set, the negation changes the overall sentiment of a sentence from positive to negative. [sent-420, score-0.575]
72 Hence, we compute accuracy in terms of correct sentiment reversal from positive to negative. [sent-421, score-0.544]
73 9 shows two examples of positive negation the RNTN correctly classified, even if negation is less obvious in the case of ‘least’ . [sent-423, score-0.37]
74 Table 2 (left) gives the accuracies over 21 positive sentences and their negation for all models. [sent-424, score-0.236]
75 The RNTN has the highest reversal accuracy, showing its ability to structurally learn negation of posi- tive sentences. [sent-425, score-0.171]
76 But what if the model simply makes phrases very negative when negation is in the sentence? [sent-426, score-0.329]
77 When negative sentences are negated, the sentiment treebank shows that overall sentiment should become less negative, but not necessarily positive. [sent-430, score-0.839]
78 For instance, ‘The movie was terrible’ is negative but the ‘The movie was not terrible’ says only that it was less bad than a terrible one, not that it was good (Horn, 1989; Israel, 2001). [sent-431, score-0.439]
79 Hence, we evaluate ac- Figure 9: RNTN prediction of positive and negative (bottom right) Model sentences and their negation. [sent-432, score-0.199]
80 Negated negative is measured as increases in positive activations. [sent-443, score-0.199]
81 curacy in terms of how often each model was able to increase non-negative activation in the sentiment of the sentence. [sent-444, score-0.403]
82 9 (bottom right) shows a typical case in which sentiment was made more positive by switching the main class from negative to neutral even though both not and dull were negative. [sent-448, score-0.663]
83 It decreases positive sentiment more when it is negated and learns that negating negative phrases (such as not terrible) should increase neutral and positive activations. [sent-618, score-0.958]
84 age positive activation (for set 1) and positive values mean an increase in average positive activation (set 2). [sent-619, score-0.434]
85 Therefore we can conclude that the RNTN is best able to identify the effect of negations upon both positive and negative sentiment sentences. [sent-621, score-0.538]
86 ; be the worst special-effects creation of the year Table 3: Examples of n-grams for which the RNTN predicted the most positive and most negative responses. [sent-625, score-0.237]
87 Figure 10: Average ground truth sentiment of top 10 most positive n-grams at various n. [sent-656, score-0.441]
88 The RNTN correctly picks the more negative and positive examples. [sent-657, score-0.199]
89 5 Model Analysis: Most Positive and Negative Phrases We queried the model for its predictions on what the most positive or negative n-grams are, measured as the highest activation of the most negative and most positive classes. [sent-659, score-0.462]
90 10 shows that the RNTN selects more strongly positive phrases at most n-gram lengths compared to other models. [sent-662, score-0.2]
91 The combination of new model and data results in a system for single sentence sentiment detection that pushes state of the art by 5. [sent-665, score-0.378]
92 7% accuracy on fine-grained sentiment prediction across all phrases and captures negation of different sentiments and scope more accurately than previous models. [sent-669, score-0.691]
93 A unified architecture for natural language processing: deep neural networks with multitask learning. [sent-714, score-0.202]
94 Experimental support for a categorical compositional distributional model of meaning. [sent-739, score-0.19]
95 Dependency tree-based sentiment classification using CRFs with hidden variables. [sent-826, score-0.379]
96 Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. [sent-838, score-0.339]
97 United we stand: Improving sentiment analysis by joining machine learning and rule based methods. [sent-881, score-0.339]
98 Learning continuous phrase representations and syntactic parsing with recursive neural networks. [sent-902, score-0.342]
99 A system for real-time twitter sentiment analysis of 2012 u. [sent-958, score-0.339]
100 Large vocabulary speech recognition using deep tensor neural networks. [sent-978, score-0.371]
wordName wordTfidf (topN-words)
[('rntn', 0.553), ('sentiment', 0.339), ('tensor', 0.219), ('recursive', 0.189), ('rnn', 0.176), ('socher', 0.148), ('cb', 0.147), ('compositional', 0.142), ('compositionality', 0.134), ('negation', 0.134), ('movie', 0.124), ('neural', 0.112), ('positive', 0.102), ('phrases', 0.098), ('negative', 0.097), ('slider', 0.092), ('vectors', 0.091), ('neutral', 0.088), ('softmax', 0.087), ('node', 0.082), ('negated', 0.077), ('bag', 0.076), ('rnns', 0.074), ('parent', 0.065), ('grained', 0.064), ('activation', 0.064), ('treebank', 0.064), ('terrible', 0.064), ('composition', 0.063), ('derivative', 0.059), ('accurately', 0.056), ('binb', 0.055), ('masterful', 0.055), ('negating', 0.055), ('stanford', 0.052), ('powerful', 0.051), ('networks', 0.05), ('vector', 0.05), ('network', 0.049), ('distributional', 0.048), ('fine', 0.048), ('slice', 0.047), ('tv', 0.047), ('nb', 0.047), ('pang', 0.045), ('longer', 0.044), ('nonlinearity', 0.044), ('wonderful', 0.044), ('quantum', 0.044), ('goller', 0.044), ('rd', 0.043), ('grefenstette', 0.042), ('yessenalina', 0.041), ('representations', 0.041), ('somewhat', 0.04), ('classification', 0.04), ('tree', 0.04), ('deep', 0.04), ('pushes', 0.039), ('worst', 0.038), ('backprop', 0.037), ('dull', 0.037), ('forgettable', 0.037), ('holographic', 0.037), ('moilanen', 0.037), ('painfully', 0.037), ('polanyi', 0.037), ('ranzato', 0.037), ('reversal', 0.037), ('rntns', 0.037), ('vecavg', 0.037), ('tanh', 0.037), ('accuracy', 0.036), ('analyze', 0.035), ('bottom', 0.034), ('capture', 0.034), ('fashion', 0.034), ('spaces', 0.034), ('matrix', 0.033), ('children', 0.033), ('pado', 0.032), ('jenatton', 0.032), ('mess', 0.032), ('pollack', 0.032), ('rentoumi', 0.032), ('uchler', 0.032), ('dev', 0.032), ('hence', 0.031), ('logical', 0.03), ('bad', 0.03), ('compute', 0.03), ('memories', 0.029), ('nakagawa', 0.029), ('snyder', 0.029), ('ias', 0.029), ('pulman', 0.029), ('contrastive', 0.029), ('parse', 0.029), ('labels', 0.028), ('scope', 0.028)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999976 158 emnlp-2013-Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Author: Richard Socher ; Alex Perelygin ; Jean Wu ; Jason Chuang ; Christopher D. Manning ; Andrew Ng ; Christopher Potts
Abstract: Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.
2 0.28148776 143 emnlp-2013-Open Domain Targeted Sentiment
Author: Margaret Mitchell ; Jacqui Aguilar ; Theresa Wilson ; Benjamin Van Durme
Abstract: We propose a novel approach to sentiment analysis for a low resource setting. The intuition behind this work is that sentiment expressed towards an entity, targeted sentiment, may be viewed as a span of sentiment expressed across the entity. This representation allows us to model sentiment detection as a sequence tagging problem, jointly discovering people and organizations along with whether there is sentiment directed towards them. We compare performance in both Spanish and English on microblog data, using only a sentiment lexicon as an external resource. By leveraging linguisticallyinformed features within conditional random fields (CRFs) trained to minimize empirical risk, our best models in Spanish significantly outperform a strong baseline, and reach around 90% accuracy on the combined task of named entity recognition and sentiment prediction. Our models in English, trained on a much smaller dataset, are not yet statistically significant against their baselines.
3 0.25808856 172 emnlp-2013-Simple Customization of Recursive Neural Networks for Semantic Relation Classification
Author: Kazuma Hashimoto ; Makoto Miwa ; Yoshimasa Tsuruoka ; Takashi Chikayama
Abstract: In this paper, we present a recursive neural network (RNN) model that works on a syntactic tree. Our model differs from previous RNN models in that the model allows for an explicit weighting of important phrases for the target task. We also propose to average parameters in training. Our experimental results on semantic relation classification show that both phrase categories and task-specific weighting significantly improve the prediction accuracy of the model. We also show that averaging the model parameters is effective in stabilizing the learning and improves generalization capacity. The proposed model marks scores competitive with state-of-the-art RNN-based models.
4 0.17810172 109 emnlp-2013-Is Twitter A Better Corpus for Measuring Sentiment Similarity?
Author: Shi Feng ; Le Zhang ; Binyang Li ; Daling Wang ; Ge Yu ; Kam-Fai Wong
Abstract: Extensive experiments have validated the effectiveness of the corpus-based method for classifying the word’s sentiment polarity. However, no work is done for comparing different corpora in the polarity classification task. Nowadays, Twitter has aggregated huge amount of data that are full of people’s sentiments. In this paper, we empirically evaluate the performance of different corpora in sentiment similarity measurement, which is the fundamental task for word polarity classification. Experiment results show that the Twitter data can achieve a much better performance than the Google, Web1T and Wikipedia based methods.
5 0.16227175 154 emnlp-2013-Prior Disambiguation of Word Tensors for Constructing Sentence Vectors
Author: Dimitri Kartsaklis ; Mehrnoosh Sadrzadeh
Abstract: Recent work has shown that compositionaldistributional models using element-wise operations on contextual word vectors benefit from the introduction of a prior disambiguation step. The purpose of this paper is to generalise these ideas to tensor-based models, where relational words such as verbs and adjectives are represented by linear maps (higher order tensors) acting on a number of arguments (vectors). We propose disambiguation algorithms for a number of tensor-based models, which we then test on a variety of tasks. The results show that disambiguation can provide better compositional representation even for the case of tensor-based models. Further- more, we confirm previous findings regarding the positive effect of disambiguation on vector mixture models, and we compare the effectiveness of the two approaches.
6 0.15854031 157 emnlp-2013-Recursive Autoencoders for ITG-Based Translation
7 0.13936873 134 emnlp-2013-Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks
8 0.13750747 137 emnlp-2013-Multi-Relational Latent Semantic Analysis
9 0.13227081 163 emnlp-2013-Sarcasm as Contrast between a Positive Sentiment and Negative Situation
10 0.12625077 60 emnlp-2013-Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models
11 0.12326594 87 emnlp-2013-Fish Transporters and Miracle Homes: How Compositional Distributional Semantics can Help NP Parsing
12 0.11390953 81 emnlp-2013-Exploring Demographic Language Variations to Improve Multilingual Sentiment Analysis in Social Media
13 0.11318479 169 emnlp-2013-Semi-Supervised Representation Learning for Cross-Lingual Text Classification
14 0.10523276 59 emnlp-2013-Deriving Adjectival Scales from Continuous Space Word Representations
15 0.10115843 191 emnlp-2013-Understanding and Quantifying Creativity in Lexical Composition
16 0.095122665 120 emnlp-2013-Learning Latent Word Representations for Domain Adaptation using Supervised Word Clustering
17 0.093925245 144 emnlp-2013-Opinion Mining in Newspaper Articles by Entropy-Based Word Connections
18 0.087082475 170 emnlp-2013-Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet
19 0.083090335 56 emnlp-2013-Deep Learning for Chinese Word Segmentation and POS Tagging
20 0.077635817 64 emnlp-2013-Discriminative Improvements to Distributional Sentence Similarity
topicId topicWeight
[(0, -0.241), (1, 0.046), (2, -0.194), (3, -0.229), (4, 0.147), (5, 0.139), (6, -0.042), (7, -0.233), (8, -0.16), (9, 0.145), (10, 0.146), (11, 0.06), (12, -0.154), (13, -0.066), (14, -0.037), (15, -0.1), (16, 0.049), (17, -0.111), (18, 0.005), (19, 0.044), (20, 0.105), (21, -0.081), (22, -0.051), (23, -0.049), (24, 0.108), (25, 0.051), (26, 0.078), (27, -0.088), (28, 0.101), (29, -0.04), (30, 0.03), (31, -0.085), (32, -0.065), (33, 0.074), (34, 0.032), (35, -0.021), (36, -0.069), (37, 0.078), (38, 0.049), (39, 0.028), (40, -0.051), (41, -0.005), (42, -0.116), (43, -0.061), (44, -0.024), (45, -0.025), (46, 0.033), (47, -0.039), (48, 0.027), (49, 0.039)]
simIndex simValue paperId paperTitle
same-paper 1 0.96437472 158 emnlp-2013-Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Author: Richard Socher ; Alex Perelygin ; Jean Wu ; Jason Chuang ; Christopher D. Manning ; Andrew Ng ; Christopher Potts
Abstract: Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.
2 0.74876601 172 emnlp-2013-Simple Customization of Recursive Neural Networks for Semantic Relation Classification
Author: Kazuma Hashimoto ; Makoto Miwa ; Yoshimasa Tsuruoka ; Takashi Chikayama
Abstract: In this paper, we present a recursive neural network (RNN) model that works on a syntactic tree. Our model differs from previous RNN models in that the model allows for an explicit weighting of important phrases for the target task. We also propose to average parameters in training. Our experimental results on semantic relation classification show that both phrase categories and task-specific weighting significantly improve the prediction accuracy of the model. We also show that averaging the model parameters is effective in stabilizing the learning and improves generalization capacity. The proposed model marks scores competitive with state-of-the-art RNN-based models.
3 0.64964616 134 emnlp-2013-Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks
Author: Masashi Tsubaki ; Kevin Duh ; Masashi Shimbo ; Yuji Matsumoto
Abstract: We present a novel vector space model for semantic co-compositionality. Inspired by Generative Lexicon Theory (Pustejovsky, 1995), our goal is a compositional model where both predicate and argument are allowed to modify each others’ meaning representations while generating the overall semantics. This readily addresses some major challenges with current vector space models, notably the polysemy issue and the use of one representation per word type. We implement cocompositionality using prototype projections on predicates/arguments and show that this is effective in adapting their word representations. We further cast the model as a neural network and propose an unsupervised algorithm to jointly train word representations with co-compositionality. The model achieves the best result to date (ρ = 0.47) on the semantic similarity task of transitive verbs (Grefenstette and Sadrzadeh, 2011).
4 0.62159717 143 emnlp-2013-Open Domain Targeted Sentiment
Author: Margaret Mitchell ; Jacqui Aguilar ; Theresa Wilson ; Benjamin Van Durme
Abstract: We propose a novel approach to sentiment analysis for a low resource setting. The intuition behind this work is that sentiment expressed towards an entity, targeted sentiment, may be viewed as a span of sentiment expressed across the entity. This representation allows us to model sentiment detection as a sequence tagging problem, jointly discovering people and organizations along with whether there is sentiment directed towards them. We compare performance in both Spanish and English on microblog data, using only a sentiment lexicon as an external resource. By leveraging linguisticallyinformed features within conditional random fields (CRFs) trained to minimize empirical risk, our best models in Spanish significantly outperform a strong baseline, and reach around 90% accuracy on the combined task of named entity recognition and sentiment prediction. Our models in English, trained on a much smaller dataset, are not yet statistically significant against their baselines.
5 0.58644879 191 emnlp-2013-Understanding and Quantifying Creativity in Lexical Composition
Author: Polina Kuznetsova ; Jianfu Chen ; Yejin Choi
Abstract: Why do certain combinations of words such as “disadvantageous peace ” or “metal to the petal” appeal to our minds as interesting expressions with a sense of creativity, while other phrases such as “quiet teenager”, or “geometrical base ” not as much? We present statistical explorations to understand the characteristics of lexical compositions that give rise to the perception of being original, interesting, and at times even artistic. We first examine various correlates of perceived creativity based on information theoretic measures and the connotation of words, then present experiments based on supervised learning that give us further insights on how different aspects of lexical composition collectively contribute to the perceived creativity.
6 0.57685941 163 emnlp-2013-Sarcasm as Contrast between a Positive Sentiment and Negative Situation
7 0.5414747 154 emnlp-2013-Prior Disambiguation of Word Tensors for Constructing Sentence Vectors
8 0.51281863 60 emnlp-2013-Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models
9 0.51062417 59 emnlp-2013-Deriving Adjectival Scales from Continuous Space Word Representations
10 0.49928787 144 emnlp-2013-Opinion Mining in Newspaper Articles by Entropy-Based Word Connections
11 0.49040315 109 emnlp-2013-Is Twitter A Better Corpus for Measuring Sentiment Similarity?
12 0.48784891 87 emnlp-2013-Fish Transporters and Miracle Homes: How Compositional Distributional Semantics can Help NP Parsing
13 0.48180008 157 emnlp-2013-Recursive Autoencoders for ITG-Based Translation
14 0.47980586 170 emnlp-2013-Sentiment Analysis: How to Derive Prior Polarities from SentiWordNet
15 0.45624182 196 emnlp-2013-Using Crowdsourcing to get Representations based on Regular Expressions
16 0.44923156 137 emnlp-2013-Multi-Relational Latent Semantic Analysis
17 0.44575325 177 emnlp-2013-Studying the Recursive Behaviour of Adjectival Modification with Compositional Distributional Semantics
18 0.39190316 81 emnlp-2013-Exploring Demographic Language Variations to Improve Multilingual Sentiment Analysis in Social Media
19 0.35513541 169 emnlp-2013-Semi-Supervised Representation Learning for Cross-Lingual Text Classification
20 0.31105322 47 emnlp-2013-Collective Opinion Target Extraction in Chinese Microblogs
topicId topicWeight
[(3, 0.038), (6, 0.298), (18, 0.034), (22, 0.037), (30, 0.099), (43, 0.016), (50, 0.026), (51, 0.159), (66, 0.035), (71, 0.034), (75, 0.039), (77, 0.039), (90, 0.015), (96, 0.025)]
simIndex simValue paperId paperTitle
1 0.82716799 134 emnlp-2013-Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks
Author: Masashi Tsubaki ; Kevin Duh ; Masashi Shimbo ; Yuji Matsumoto
Abstract: We present a novel vector space model for semantic co-compositionality. Inspired by Generative Lexicon Theory (Pustejovsky, 1995), our goal is a compositional model where both predicate and argument are allowed to modify each others’ meaning representations while generating the overall semantics. This readily addresses some major challenges with current vector space models, notably the polysemy issue and the use of one representation per word type. We implement cocompositionality using prototype projections on predicates/arguments and show that this is effective in adapting their word representations. We further cast the model as a neural network and propose an unsupervised algorithm to jointly train word representations with co-compositionality. The model achieves the best result to date (ρ = 0.47) on the semantic similarity task of transitive verbs (Grefenstette and Sadrzadeh, 2011).
same-paper 2 0.79447669 158 emnlp-2013-Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Author: Richard Socher ; Alex Perelygin ; Jean Wu ; Jason Chuang ; Christopher D. Manning ; Andrew Ng ; Christopher Potts
Abstract: Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.
3 0.62427127 87 emnlp-2013-Fish Transporters and Miracle Homes: How Compositional Distributional Semantics can Help NP Parsing
Author: Angeliki Lazaridou ; Eva Maria Vecchi ; Marco Baroni
Abstract: In this work, we argue that measures that have been shown to quantify the degree of semantic plausibility of phrases, as obtained from their compositionally-derived distributional semantic representations, can resolve syntactic ambiguities. We exploit this idea to choose the correct parsing of NPs (e.g., (live fish) transporter rather than live (fish transporter)). We show that our plausibility cues outperform a strong baseline and significantly improve performance when used in combination with state-of-the-art features.
4 0.62255359 157 emnlp-2013-Recursive Autoencoders for ITG-Based Translation
Author: Peng Li ; Yang Liu ; Maosong Sun
Abstract: While inversion transduction grammar (ITG) is well suited for modeling ordering shifts between languages, how to make applying the two reordering rules (i.e., straight and inverted) dependent on actual blocks being merged remains a challenge. Unlike previous work that only uses boundary words, we propose to use recursive autoencoders to make full use of the entire merging blocks alternatively. The recursive autoencoders are capable of generating vector space representations for variable-sized phrases, which enable predicting orders to exploit syntactic and semantic information from a neural language modeling’s perspective. Experiments on the NIST 2008 dataset show that our system significantly improves over the MaxEnt classifier by 1.07 BLEU points.
5 0.6206761 172 emnlp-2013-Simple Customization of Recursive Neural Networks for Semantic Relation Classification
Author: Kazuma Hashimoto ; Makoto Miwa ; Yoshimasa Tsuruoka ; Takashi Chikayama
Abstract: In this paper, we present a recursive neural network (RNN) model that works on a syntactic tree. Our model differs from previous RNN models in that the model allows for an explicit weighting of important phrases for the target task. We also propose to average parameters in training. Our experimental results on semantic relation classification show that both phrase categories and task-specific weighting significantly improve the prediction accuracy of the model. We also show that averaging the model parameters is effective in stabilizing the learning and improves generalization capacity. The proposed model marks scores competitive with state-of-the-art RNN-based models.
6 0.60842073 154 emnlp-2013-Prior Disambiguation of Word Tensors for Constructing Sentence Vectors
7 0.58673286 38 emnlp-2013-Bilingual Word Embeddings for Phrase-Based Machine Translation
9 0.58396399 60 emnlp-2013-Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models
10 0.5802418 191 emnlp-2013-Understanding and Quantifying Creativity in Lexical Composition
11 0.57134461 156 emnlp-2013-Recurrent Continuous Translation Models
12 0.57108229 56 emnlp-2013-Deep Learning for Chinese Word Segmentation and POS Tagging
13 0.56104761 15 emnlp-2013-A Systematic Exploration of Diversity in Machine Translation
14 0.56076461 175 emnlp-2013-Source-Side Classifier Preordering for Machine Translation
15 0.56022692 143 emnlp-2013-Open Domain Targeted Sentiment
16 0.55948508 107 emnlp-2013-Interactive Machine Translation using Hierarchical Translation Models
17 0.55851144 48 emnlp-2013-Collective Personal Profile Summarization with Social Networks
18 0.55690175 53 emnlp-2013-Cross-Lingual Discriminative Learning of Sequence Models with Posterior Regularization
19 0.55607969 114 emnlp-2013-Joint Learning and Inference for Grammatical Error Correction
20 0.55606896 47 emnlp-2013-Collective Opinion Target Extraction in Chinese Microblogs