emnlp emnlp2012 emnlp2012-49 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Keith Stevens ; Philip Kegelmeyer ; David Andrzejewski ; David Buttler
Abstract: We apply two new automated semantic evaluations to three distinct latent topic models. Both metrics have been shown to align with human evaluations and provide a balance between internal measures of information gain and comparisons to human ratings of coherent topics. We improve upon the measures by introducing new aggregate measures that allows for comparing complete topic models. We further compare the automated measures to other metrics for topic models, comparison to manually crafted semantic tests and document classification. Our experiments reveal that LDA and LSA each have different strengths; LDA best learns descriptive topics while LSA is best at creating a compact semantic representation ofdocuments and words in a corpus.
Reference: text
sentIndex sentText sentNum sentScore
1 gov , , Abstract We apply two new automated semantic evaluations to three distinct latent topic models. [sent-3, score-0.507]
2 Both metrics have been shown to align with human evaluations and provide a balance between internal measures of information gain and comparisons to human ratings of coherent topics. [sent-4, score-0.201]
3 We improve upon the measures by introducing new aggregate measures that allows for comparing complete topic models. [sent-5, score-0.424]
4 We further compare the automated measures to other metrics for topic models, comparison to manually crafted semantic tests and document classification. [sent-6, score-0.505]
5 Our experiments reveal that LDA and LSA each have different strengths; LDA best learns descriptive topics while LSA is best at creating a compact semantic representation ofdocuments and words in a corpus. [sent-7, score-0.443]
6 Based on the words used within a document, they mine topic level relations by assuming that a single document covers a small set of concise topics. [sent-9, score-0.334]
7 Once learned, these topics often correlate well with human concepts. [sent-10, score-0.321]
8 For example, one model might produce topics that cover ideas such as government affairs, sports, and movies. [sent-11, score-0.321]
9 952 When using a topic model, we are primarily concerned with the degree to which the learned topics match human judgments and help us differentiate between ideas. [sent-13, score-0.692]
10 Evaluations have ranged from fully automated intrinsic evaluations to manually crafted extrinsic evaluations. [sent-15, score-0.184]
11 Previous extrinsic evaluations have used the learned topics to compactly represent a small fixed vocabulary and compared this distributional space to human judgments of similarity (Jurgens and Stevens, 2010). [sent-16, score-0.531]
12 Conversely, intrinsic measures have evaluated the amount of information encoded by the topics, where perplexity is one common example(Wallach et al. [sent-18, score-0.086]
13 (2009) found that these intrinsic measures do not always correlate with semantically interpretable topics. [sent-20, score-0.086]
14 Furthermore, few evaluations have used the same metrics to compare distinct approaches such as Latent Dirichlet Allocation (LDA) (Blei et al. [sent-21, score-0.085]
15 We now provide a comprehensive and automated evaluation of these three distinct models (LDA, LSA, NMF), for automatically learning semantic topics. [sent-24, score-0.084]
16 For our evaluation, we use two recent automated coherence measures (Mimno et al. [sent-26, score-0.366]
17 lc L2a0n1g2ua Agseso Pcrioactieosnsi fnogr a Cnodm Cpoumtaptiuotna tilo Lnianlg Nuaist uircasl originally designed for LDA that bridge the gap between comparisons to human judgments and intrinsic measures such as perplexity. [sent-30, score-0.123]
18 How do these topics relate to often used semantic tests? [sent-36, score-0.361]
19 We begin by summarizing the three topic models and highlighting their key differences. [sent-39, score-0.295]
20 2 Topic Models We evaluate three latent factor models that have seen widespread usage: 1. [sent-42, score-0.093]
21 We consider both LSA models as topic models as they have been used in a variety of similar contexts such as distributional similarity (Jurgens and Stevens, 2010) and word sense induction (Van de Cruys and Apidianaki, 2011; Brody and Lapata, 2009). [sent-46, score-0.367]
22 We distill the different models into a shared representation consisting of two sets of learned relations: how words interact with topics and how topics interact with documents. [sent-48, score-0.749]
23 n Footre a at choerspe u rsel watitihon Ds i dno tceurmmse otfs aTn topics as × (1) a V T matrix, W, that indicates the strength ae Vach × w Tor md ahtarsix i,n W Wea,c hth topic, iacnadte (2) a T D matrix, H, that indicates the strength ae Tach × topic haatrsi xin, Hea,c hth daotc iundmiceantte. [sent-50, score-0.708]
24 It first assumes that there are a fixed set of topics, T used throughout the corpus, ea and fi xeaedch s topic z i sc sa,s Tso ucsiaetedd th wroiuthg a omutul thtienomial distribution over the vocabulary Φz, which is drawn from a Dirichlet prior Dir(β). [sent-55, score-0.295]
25 Choose Θi ∼ Dir(α), a topic distribution for Di 2. [sent-57, score-0.295]
26 For each word wj ∈ Di: (a) Select a topic zj ∼ Θi (b) Select the word wj ∼ Φzj In this model, the Θ distributions represent the probability of each topic appearing in each document and the Φ distributions represent the probability of words being used for each topic. [sent-58, score-0.715]
27 , 1998) learns topics by first forming a traditional term by document matrix used in information retrieval and then smoothing the counts to enhance the weight of informative words. [sent-64, score-0.455]
28 LSA then decomposes this smoothed, term by document matrix in order to generalize observed relations between words and documents. [sent-66, score-0.124]
29 10 words from several high and low quality topics when ordered by the UCI Coherence Topic labels were chosen in an ad hoc manner only to briefly summarize the topic’s focus. [sent-92, score-0.382]
30 We later refer to these two LSA models simply as SVD and NMF to emphasize the difference in factorization method. [sent-95, score-0.084]
31 how well they remove noise, which is encoded by the diagonal singular value matrix Σ. [sent-100, score-0.099]
32 These measurements help distinguish between topics that are semantically interpretable topics and topics that are artifacts of statistical inference, see Table 1 for examples ordered by the UCI measure. [sent-108, score-0.963]
33 For our evaluations, we consider two new coherence measures designed for LDA, both of which have been shown to match well with human judgements of topic quality: (1) The UCI measure (Newman et al. [sent-109, score-0.663]
34 Both measures compute the coherence of a topic as the sum of pairwise distributional similarity 3We note that the alternative KL-Divergence form of NMF has been directly linked to PLSA (Ding et al. [sent-112, score-0.689]
35 ) (vi,Xvj)∈V where V is a set of word describing the topic and ? [sent-115, score-0.295]
36 indicates a smoothing factor which guarantees that score returns real numbers. [sent-116, score-0.085]
37 Significantly, the UMass metric computes these counts over the original corpus used to train the topic models, rather than an external corpus. [sent-129, score-0.295]
38 4 Evaluation We evaluate the quality of our three topic models (LDA, SVD, and NMF) with three experiments. [sent-132, score-0.333]
39 We focus first on evaluating aggregate coherence methods for a complete topic model and consider the differences between each model as we learn an increasing number of topics. [sent-133, score-0.638]
40 Secondly, we compare coherence scores to previous semantic evaluations. [sent-134, score-0.326]
41 955 Lastly, we use the learned topics in a classification task and evaluate whether or not coherent topics are equally informative when discriminating between documents. [sent-135, score-0.771]
42 In all experiments, we compute the coherence with the top 10 words from each topic that had the highest weight, in terms of LDA and NMF this corresponds with a high probability of the term describing the topic but for SVD there is no clear semantic interpretation. [sent-141, score-0.916]
43 1 Aggregate methods for topic coherence Before we can compare topic models, we require an aggregate measure that represents the quality of a complete model, rather than individual topics. [sent-143, score-0.971]
44 We consider two aggregates methods: (1) the average coherence of all topics and (2) the entropy of the coherence for all topics. [sent-144, score-0.934]
45 The average coherence provides a quick summarization of a model’s quality whereas the entropy provides an alternate summarization that differentiates between two interesting situations. [sent-145, score-0.365]
46 Since entropy measures the complexity of a probability distribution, it can easily differentiate between uniform distributions and multimodal, distributions. [sent-146, score-0.108]
47 This distinction is relevant when users prefer to have roughly uniform topic quality instead of a wide gap between high- and low-quality topics, or vice versa. [sent-147, score-0.333]
48 Figure 1 shows the average coherence scores for each model as we vary the number of topics. [sent-150, score-0.286]
49 While the entropy for the UMass score stays stable for all models, NMF produces erratic entropy results under the UCI score as we learn more topics. [sent-156, score-0.132]
50 As entropy is higher for even distributions and lower for all other distributions, these results suggest that the NMF is learning topics with drastically different levels of quality, i. [sent-157, score-0.393]
51 some with high quality and some with very low quality, but the average coherence over all topics do not account for this. [sent-159, score-0.645]
52 Low quality topics may be composed of highly unrelated words that can’t be fit into another topic, and in this case, our smoothing factor, ? [sent-160, score-0.44]
53 , may be ar956 tificially increasing the score for unrelated words. [sent-161, score-0.073]
54 = 10−12, which should significantly reduce the score for completely unrelated words. [sent-165, score-0.073]
55 Here, we see a significant change in the performance of NMF, the average coherence decreases dramatically as we learn more topics. [sent-166, score-0.286]
56 In figure 4 we lastly compute the average coherence using only the top 10% most coherence topics with ? [sent-168, score-0.922]
57 = 10−12 of topics having a low coherence, NMF appears to be learning more low quality topics once it’s learned the first 100 topics, whereas LDA learns fewer low quality topics in general. [sent-173, score-1.078]
58 2 Word Similarity Tasks The initial evaluations for each coherence measure asked human judges to directly evaluate topics (Newman et al. [sent-175, score-0.693]
59 We expand upon this comparison to human judgments by considering word similarity tasks that have often been used to evaluate distributional semantic spaces (Jurgens and Stevens, 2010). [sent-178, score-0.149]
60 Here, we use the learned topics as generalized semantics describ- 957 ing our knowledge about words. [sent-179, score-0.36]
61 If a model’s topics generalize the knowledge accurately, we would expect similar words, such as “cat” and “dog”, to be represented with a similar set of topics. [sent-180, score-0.321]
62 Rather than evaluating individual topics, this similarity task considers the knowledge within the entire set of topics, the topics act as more compact representation for the known words in a corpus. [sent-181, score-0.384]
63 In each task, human judges were asked to evaluate the similarity or relatedness between different sets of word pairs. [sent-184, score-0.094]
64 0210 20 30 40 50 modSNLeVMDl AF Topics (b) UCI Figure 7: Correlation between topic coherence and topic ranking in classification of words and asked to rate their similarity on a scale from 0 to 4, where a higher score indicates a more similar word pair. [sent-205, score-1.002]
65 (2002) broadens the word similarity evaluation and asked 13 to 16 different subjects to rate 353 word pairs on a scale from 0 to 10 based on their relatedness, where relatedness includes similarity and other semantic relations. [sent-207, score-0.174]
66 We can evaluate each topic model by computing the cosine similarity between each pair of words in the evaluate set and then compare the model’s ratings to the human ratings by ranked correlation. [sent-208, score-0.389]
67 A high correlation signifies that the topics closely model human judgments. [sent-209, score-0.35]
68 While our first experiment showed that SVD was the worst model in terms of topic coherence scores, this experiment indicates that SVD provides an accurate, stable, and reliable approximation to human judgements of similarity and relatedness between word pairs in comparison to other topic models. [sent-213, score-0.992]
69 3 Coherence versus Classification For our final experiment, we examine the relationship between topic coherence and classification accuracy for each topic model. [sent-215, score-0.913]
70 We address this question by performing a document classification task using the topic representations of documents as input features and examine the relationship between topic coherence and the usefulness of the corre- sponding feature for classification. [sent-231, score-0.976]
71 We trained each topic model with all 92,600 New York Times articles as before. [sent-232, score-0.326]
72 We use the section labels provided for each article as class labels, where each label indicates the on-line section(s) under which the article was published and should thus be related to the topics contained in each article. [sent-233, score-0.345]
73 To reduce the noise in our data set we narrow down the articles to those that have only one label and whose 959 label is applied to at least 2000 documents. [sent-234, score-0.079]
74 This results in 57,696 articles with label distributions listed in Table 2. [sent-235, score-0.086]
75 We then represent each document using columns in the topic by document matrix H learned for each topic model. [sent-236, score-0.769]
76 3675 Table 2: Section label counts for New York Times articles used for classification For each topic model trained on N topics, we performed stratified 10-fold cross-validation on the 57,696 labeled articles. [sent-239, score-0.387]
77 We evaluate the strength of each topic during classification by tracking the number of times each node in our decision trees observe each topic, please see (Caruana et al. [sent-247, score-0.332]
78 Figure 8 plot the relationship between this feature ranking and the topic coherence for each topic when training LDA, SVD, and NMF on 500 topics. [sent-249, score-0.876]
79 Most topics for each model provide little classification information, but SVD shows a much higher rank for several topics with a relatively higher coherence score. [sent-250, score-0.993]
80 Interestingly, for all models, the most coherent topics are not the most informative. [sent-251, score-0.374]
81 Figure 7 plots a more compact view of this same relationship: the Spearman rank correlation between classification feature rank and topic coherence. [sent-252, score-0.44]
82 NMF shows the highest correlation between rank and coherence, but none of the models show a high correlation when using more than 100 topics. [sent-253, score-0.086]
83 SVD has the lowest correlation, which is probably due to the model’s overall low coherence yet high classification accuracy. [sent-254, score-0.323]
84 First, we discovered that the coherence metrics depend heavily on the smoothing factor ? [sent-256, score-0.369]
85 We suspect that this was not an issue in previous studies with the coherence measures as LDA prefers to form topics from words that co-occur frequently, whereas NMF and SVD have no such preferences and often create low quality topics from completely unrelated words. [sent-261, score-1.05]
86 We also found that the UCI measure often agreed with the UMass measure, but the UCI-entropy aggregate method induced more separation between LSA, SVD, and NMF in terms of topic coherence. [sent-264, score-0.352]
87 This measure also revealed the importance of the smoothing factor for topic coherence measures. [sent-265, score-0.641]
88 With respects to human judgements, we found that coherence scores do not always indicate a bet960 ter representation of distributional information. [sent-266, score-0.318]
89 The SVD model consistently out performed both LDA and NMF models, which each had higher coherence scores, when attempting to predict human judgements of similarity. [sent-267, score-0.332]
90 Lastly, we found all models capable of producing topics that improved document classification. [sent-268, score-0.36]
91 At the same time, SVD provided the most information during classification and outperformed the other models, which again had more coherent topics. [sent-269, score-0.09]
92 Our comparison between topic coherence scores and feature importance in classification revealed that relatively high quality topics, but not the most coherent topics, drive most of the classification decisions, and most topics do not affect the accuracy. [sent-270, score-1.067]
93 Overall, we see that each topic model paradigm has it’s own strengths and weaknesses. [sent-271, score-0.295]
94 Latent Semantic Analysis with Singular Value Decomposition fails to form individual topics that aggregate similar words, but it does remarkably well when considering all the learned topics as similar words develop a similar topic representation. [sent-272, score-1.033]
95 Conversely, both Non Negative Matrix factorization and Latent Dirichlet Allocation learn concise and coherent topics and achieved similar performance on our evaluations. [sent-274, score-0.458]
96 However, NMF learns more incoherent topics than LDA and SVD. [sent-275, score-0.352]
97 For applications in which a human end-user will interact with learned topics, the flexibility of LDA and the coherence advantages of LDA warrant strong consideration. [sent-276, score-0.359]
98 Reading tea leaves : How humans interpret topic models. [sent-323, score-0.295]
99 On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. [sent-327, score-0.252]
100 A solution to platos problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. [sent-355, score-0.106]
wordName wordTfidf (topN-words)
[('nmf', 0.554), ('topics', 0.321), ('topic', 0.295), ('coherence', 0.286), ('uci', 0.25), ('svd', 0.238), ('umass', 0.215), ('lda', 0.186), ('lsa', 0.114), ('factorization', 0.084), ('jurgens', 0.077), ('livermore', 0.072), ('rubenstein', 0.069), ('latent', 0.066), ('evaluations', 0.062), ('matrix', 0.062), ('stevens', 0.061), ('vea', 0.061), ('aggregate', 0.057), ('goodenough', 0.056), ('cruys', 0.054), ('coherent', 0.053), ('mimno', 0.051), ('intrinsic', 0.05), ('newman', 0.048), ('landauer', 0.048), ('unrelated', 0.048), ('allocation', 0.046), ('dirichlet', 0.046), ('judgements', 0.046), ('automated', 0.044), ('brody', 0.042), ('entropy', 0.041), ('semantic', 0.04), ('similarity', 0.04), ('learned', 0.039), ('document', 0.039), ('finkelstein', 0.039), ('quality', 0.038), ('singular', 0.037), ('judgments', 0.037), ('classification', 0.037), ('wallach', 0.036), ('york', 0.036), ('decomposition', 0.036), ('andrzejewski', 0.036), ('apidianaki', 0.036), ('bagged', 0.036), ('banfield', 0.036), ('caruana', 0.036), ('dutnais', 0.036), ('edmund', 0.036), ('elr', 0.036), ('grea', 0.036), ('ofdocuments', 0.036), ('pauca', 0.036), ('talley', 0.036), ('tioa', 0.036), ('measures', 0.036), ('ding', 0.034), ('interact', 0.034), ('smoothing', 0.033), ('distributional', 0.032), ('distributions', 0.031), ('unnormalized', 0.031), ('vt', 0.031), ('incoherent', 0.031), ('articles', 0.031), ('relatedness', 0.03), ('correlation', 0.029), ('lastly', 0.029), ('rank', 0.028), ('crafted', 0.028), ('sliding', 0.028), ('factor', 0.027), ('ratings', 0.027), ('blei', 0.026), ('hanna', 0.026), ('dir', 0.026), ('california', 0.025), ('ensemble', 0.025), ('score', 0.025), ('documents', 0.024), ('asked', 0.024), ('matrices', 0.024), ('zj', 0.024), ('baroni', 0.024), ('hth', 0.024), ('label', 0.024), ('compact', 0.023), ('descriptive', 0.023), ('ho', 0.023), ('reconstruction', 0.023), ('hoc', 0.023), ('decomposes', 0.023), ('metrics', 0.023), ('interestingly', 0.022), ('david', 0.022), ('eg', 0.022), ('ae', 0.022)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999994 49 emnlp-2012-Exploring Topic Coherence over Many Models and Many Topics
Author: Keith Stevens ; Philip Kegelmeyer ; David Andrzejewski ; David Buttler
Abstract: We apply two new automated semantic evaluations to three distinct latent topic models. Both metrics have been shown to align with human evaluations and provide a balance between internal measures of information gain and comparisons to human ratings of coherent topics. We improve upon the measures by introducing new aggregate measures that allows for comparing complete topic models. We further compare the automated measures to other metrics for topic models, comparison to manually crafted semantic tests and document classification. Our experiments reveal that LDA and LSA each have different strengths; LDA best learns descriptive topics while LSA is best at creating a compact semantic representation ofdocuments and words in a corpus.
2 0.21535343 90 emnlp-2012-Modelling Sequential Text with an Adaptive Topic Model
Author: Lan Du ; Wray Buntine ; Huidong Jin
Abstract: Topic models are increasingly being used for text analysis tasks, often times replacing earlier semantic techniques such as latent semantic analysis. In this paper, we develop a novel adaptive topic model with the ability to adapt topics from both the previous segment and the parent document. For this proposed model, a Gibbs sampler is developed for doing posterior inference. Experimental results show that with topic adaptation, our model significantly improves over existing approaches in terms of perplexity, and is able to uncover clear sequential structure on, for example, Herman Melville’s book “Moby Dick”.
3 0.18125322 115 emnlp-2012-SSHLDA: A Semi-Supervised Hierarchical Topic Model
Author: Xian-Ling Mao ; Zhao-Yan Ming ; Tat-Seng Chua ; Si Li ; Hongfei Yan ; Xiaoming Li
Abstract: Supervised hierarchical topic modeling and unsupervised hierarchical topic modeling are usually used to obtain hierarchical topics, such as hLLDA and hLDA. Supervised hierarchical topic modeling makes heavy use of the information from observed hierarchical labels, but cannot explore new topics; while unsupervised hierarchical topic modeling is able to detect automatically new topics in the data space, but does not make use of any information from hierarchical labels. In this paper, we propose a semi-supervised hierarchical topic model which aims to explore new topics automatically in the data space while incorporating the information from observed hierarchical labels into the modeling process, called SemiSupervised Hierarchical Latent Dirichlet Allocation (SSHLDA). We also prove that hLDA and hLLDA are special cases of SSHLDA. We . conduct experiments on Yahoo! Answers and ODP datasets, and assess the performance in terms of perplexity and clustering. The experimental results show that predictive ability of SSHLDA is better than that of baselines, and SSHLDA can also achieve significant improvement over baselines for clustering on the FScore measure.
4 0.17697023 8 emnlp-2012-A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes
Author: Robert Lindsey ; William Headden ; Michael Stipicevic
Abstract: Topic models traditionally rely on the bagof-words assumption. In data mining applications, this often results in end-users being presented with inscrutable lists of topical unigrams, single words inferred as representative of their topics. In this article, we present a hierarchical generative probabilistic model of topical phrases. The model simultaneously infers the location, length, and topic of phrases within a corpus and relaxes the bagof-words assumption within phrases by using a hierarchy of Pitman-Yor processes. We use Markov chain Monte Carlo techniques for approximate inference in the model and perform slice sampling to learn its hyperparameters. We show via an experiment on human subjects that our model finds substantially better, more interpretable topical phrases than do competing models.
5 0.17434064 19 emnlp-2012-An Entity-Topic Model for Entity Linking
Author: Xianpei Han ; Le Sun
Abstract: Entity Linking (EL) has received considerable attention in recent years. Given many name mentions in a document, the goal of EL is to predict their referent entities in a knowledge base. Traditionally, there have been two distinct directions of EL research: one focusing on the effects of mention’s context compatibility, assuming that “the referent entity of a mention is reflected by its context”; the other dealing with the effects of document’s topic coherence, assuming that “a mention ’s referent entity should be coherent with the document’ ’s main topics”. In this paper, we propose a generative model called entitytopic model, to effectively join the above two complementary directions together. By jointly modeling and exploiting the context compatibility, the topic coherence and the correlation between them, our model can – accurately link all mentions in a document using both the local information (including the words and the mentions in a document) and the global knowledge (including the topic knowledge, the entity context knowledge and the entity name knowledge). Experimental results demonstrate the effectiveness of the proposed model. 1
6 0.13682361 47 emnlp-2012-Explore Person Specific Evidence in Web Person Name Disambiguation
7 0.12596534 89 emnlp-2012-Mixed Membership Markov Models for Unsupervised Conversation Modeling
8 0.11945637 3 emnlp-2012-A Coherence Model Based on Syntactic Patterns
9 0.091345869 29 emnlp-2012-Concurrent Acquisition of Word Meaning and Lexical Categories
10 0.08954455 107 emnlp-2012-Polarity Inducing Latent Semantic Analysis
11 0.074007519 33 emnlp-2012-Discovering Diverse and Salient Threads in Document Collections
12 0.071099736 48 emnlp-2012-Exploring Adaptor Grammars for Native Language Identification
13 0.066585824 16 emnlp-2012-Aligning Predicates across Monolingual Comparable Texts using Graph-based Clustering
14 0.064359777 60 emnlp-2012-Generative Goal-Driven User Simulation for Dialog Management
15 0.064009361 114 emnlp-2012-Revisiting the Predictability of Language: Response Completion in Social Media
16 0.054954909 4 emnlp-2012-A Comparison of Vector-based Representations for Semantic Composition
17 0.049677037 116 emnlp-2012-Semantic Compositionality through Recursive Matrix-Vector Spaces
18 0.047203533 20 emnlp-2012-Answering Opinion Questions on Products by Exploiting Hierarchical Organization of Consumer Reviews
19 0.046498924 32 emnlp-2012-Detecting Subgroups in Online Discussions by Modeling Positive and Negative Relations among Participants
20 0.044143587 50 emnlp-2012-Extending Machine Translation Evaluation Metrics with Lexical Cohesion to Document Level
topicId topicWeight
[(0, 0.184), (1, 0.126), (2, 0.06), (3, 0.205), (4, -0.333), (5, 0.193), (6, -0.038), (7, -0.083), (8, -0.106), (9, 0.015), (10, -0.108), (11, -0.048), (12, 0.134), (13, 0.126), (14, 0.128), (15, -0.029), (16, 0.026), (17, 0.006), (18, 0.085), (19, -0.021), (20, 0.051), (21, -0.015), (22, 0.063), (23, 0.035), (24, 0.009), (25, -0.037), (26, 0.019), (27, -0.107), (28, 0.066), (29, 0.03), (30, -0.072), (31, -0.006), (32, 0.064), (33, 0.092), (34, 0.103), (35, 0.061), (36, 0.004), (37, 0.028), (38, 0.074), (39, -0.065), (40, -0.024), (41, -0.057), (42, 0.024), (43, -0.002), (44, 0.042), (45, -0.03), (46, -0.039), (47, 0.042), (48, 0.03), (49, -0.008)]
simIndex simValue paperId paperTitle
same-paper 1 0.9775809 49 emnlp-2012-Exploring Topic Coherence over Many Models and Many Topics
Author: Keith Stevens ; Philip Kegelmeyer ; David Andrzejewski ; David Buttler
Abstract: We apply two new automated semantic evaluations to three distinct latent topic models. Both metrics have been shown to align with human evaluations and provide a balance between internal measures of information gain and comparisons to human ratings of coherent topics. We improve upon the measures by introducing new aggregate measures that allows for comparing complete topic models. We further compare the automated measures to other metrics for topic models, comparison to manually crafted semantic tests and document classification. Our experiments reveal that LDA and LSA each have different strengths; LDA best learns descriptive topics while LSA is best at creating a compact semantic representation ofdocuments and words in a corpus.
2 0.82796007 90 emnlp-2012-Modelling Sequential Text with an Adaptive Topic Model
Author: Lan Du ; Wray Buntine ; Huidong Jin
Abstract: Topic models are increasingly being used for text analysis tasks, often times replacing earlier semantic techniques such as latent semantic analysis. In this paper, we develop a novel adaptive topic model with the ability to adapt topics from both the previous segment and the parent document. For this proposed model, a Gibbs sampler is developed for doing posterior inference. Experimental results show that with topic adaptation, our model significantly improves over existing approaches in terms of perplexity, and is able to uncover clear sequential structure on, for example, Herman Melville’s book “Moby Dick”.
3 0.82327461 115 emnlp-2012-SSHLDA: A Semi-Supervised Hierarchical Topic Model
Author: Xian-Ling Mao ; Zhao-Yan Ming ; Tat-Seng Chua ; Si Li ; Hongfei Yan ; Xiaoming Li
Abstract: Supervised hierarchical topic modeling and unsupervised hierarchical topic modeling are usually used to obtain hierarchical topics, such as hLLDA and hLDA. Supervised hierarchical topic modeling makes heavy use of the information from observed hierarchical labels, but cannot explore new topics; while unsupervised hierarchical topic modeling is able to detect automatically new topics in the data space, but does not make use of any information from hierarchical labels. In this paper, we propose a semi-supervised hierarchical topic model which aims to explore new topics automatically in the data space while incorporating the information from observed hierarchical labels into the modeling process, called SemiSupervised Hierarchical Latent Dirichlet Allocation (SSHLDA). We also prove that hLDA and hLLDA are special cases of SSHLDA. We . conduct experiments on Yahoo! Answers and ODP datasets, and assess the performance in terms of perplexity and clustering. The experimental results show that predictive ability of SSHLDA is better than that of baselines, and SSHLDA can also achieve significant improvement over baselines for clustering on the FScore measure.
4 0.75177652 8 emnlp-2012-A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes
Author: Robert Lindsey ; William Headden ; Michael Stipicevic
Abstract: Topic models traditionally rely on the bagof-words assumption. In data mining applications, this often results in end-users being presented with inscrutable lists of topical unigrams, single words inferred as representative of their topics. In this article, we present a hierarchical generative probabilistic model of topical phrases. The model simultaneously infers the location, length, and topic of phrases within a corpus and relaxes the bagof-words assumption within phrases by using a hierarchy of Pitman-Yor processes. We use Markov chain Monte Carlo techniques for approximate inference in the model and perform slice sampling to learn its hyperparameters. We show via an experiment on human subjects that our model finds substantially better, more interpretable topical phrases than do competing models.
5 0.6164403 19 emnlp-2012-An Entity-Topic Model for Entity Linking
Author: Xianpei Han ; Le Sun
Abstract: Entity Linking (EL) has received considerable attention in recent years. Given many name mentions in a document, the goal of EL is to predict their referent entities in a knowledge base. Traditionally, there have been two distinct directions of EL research: one focusing on the effects of mention’s context compatibility, assuming that “the referent entity of a mention is reflected by its context”; the other dealing with the effects of document’s topic coherence, assuming that “a mention ’s referent entity should be coherent with the document’ ’s main topics”. In this paper, we propose a generative model called entitytopic model, to effectively join the above two complementary directions together. By jointly modeling and exploiting the context compatibility, the topic coherence and the correlation between them, our model can – accurately link all mentions in a document using both the local information (including the words and the mentions in a document) and the global knowledge (including the topic knowledge, the entity context knowledge and the entity name knowledge). Experimental results demonstrate the effectiveness of the proposed model. 1
6 0.52786189 89 emnlp-2012-Mixed Membership Markov Models for Unsupervised Conversation Modeling
7 0.50880295 47 emnlp-2012-Explore Person Specific Evidence in Web Person Name Disambiguation
8 0.45123431 33 emnlp-2012-Discovering Diverse and Salient Threads in Document Collections
9 0.41255981 107 emnlp-2012-Polarity Inducing Latent Semantic Analysis
10 0.40844598 3 emnlp-2012-A Coherence Model Based on Syntactic Patterns
11 0.34185356 29 emnlp-2012-Concurrent Acquisition of Word Meaning and Lexical Categories
12 0.3100934 48 emnlp-2012-Exploring Adaptor Grammars for Native Language Identification
13 0.29273209 16 emnlp-2012-Aligning Predicates across Monolingual Comparable Texts using Graph-based Clustering
14 0.28305131 50 emnlp-2012-Extending Machine Translation Evaluation Metrics with Lexical Cohesion to Document Level
15 0.2825368 60 emnlp-2012-Generative Goal-Driven User Simulation for Dialog Management
16 0.27846873 61 emnlp-2012-Grounded Models of Semantic Representation
17 0.27410182 114 emnlp-2012-Revisiting the Predictability of Language: Response Completion in Social Media
18 0.23981802 121 emnlp-2012-Supervised Text-based Geolocation Using Language Models on an Adaptive Grid
19 0.21167848 4 emnlp-2012-A Comparison of Vector-based Representations for Semantic Composition
20 0.20971142 32 emnlp-2012-Detecting Subgroups in Online Discussions by Modeling Positive and Negative Relations among Participants
topicId topicWeight
[(2, 0.012), (16, 0.024), (25, 0.027), (34, 0.057), (45, 0.036), (60, 0.066), (63, 0.094), (64, 0.014), (65, 0.014), (70, 0.011), (73, 0.014), (74, 0.036), (76, 0.042), (86, 0.018), (95, 0.442)]
simIndex simValue paperId paperTitle
1 0.89930511 28 emnlp-2012-Collocation Polarity Disambiguation Using Web-based Pseudo Contexts
Author: Yanyan Zhao ; Bing Qin ; Ting Liu
Abstract: This paper focuses on the task of collocation polarity disambiguation. The collocation refers to a binary tuple of a polarity word and a target (such as ⟨long, battery life⟩ or ⟨long, ast atratrguep⟩t) (, siunc whh aisch ⟨ ltohneg s,en btatitmeernyt l iofrei⟩en otrat ⟨iolonn gof, tshtaer polarity wwohirdch (“long”) changes along owniothf different targets (“battery life” or “startup”). To disambiguate a collocation’s polarity, previous work always turned to investigate the polarities of its surrounding contexts, and then assigned the majority polarity to the collocation. However, these contexts are limited, thus the resulting polarity is insufficient to be reliable. We therefore propose an unsupervised three-component framework to expand some pseudo contexts from web, to help disambiguate a collocation’s polarity.Without using any additional labeled data, experiments , show that our method is effective.
same-paper 2 0.89328033 49 emnlp-2012-Exploring Topic Coherence over Many Models and Many Topics
Author: Keith Stevens ; Philip Kegelmeyer ; David Andrzejewski ; David Buttler
Abstract: We apply two new automated semantic evaluations to three distinct latent topic models. Both metrics have been shown to align with human evaluations and provide a balance between internal measures of information gain and comparisons to human ratings of coherent topics. We improve upon the measures by introducing new aggregate measures that allows for comparing complete topic models. We further compare the automated measures to other metrics for topic models, comparison to manually crafted semantic tests and document classification. Our experiments reveal that LDA and LSA each have different strengths; LDA best learns descriptive topics while LSA is best at creating a compact semantic representation ofdocuments and words in a corpus.
3 0.88603735 83 emnlp-2012-Lexical Differences in Autobiographical Narratives from Schizophrenic Patients and Healthy Controls
Author: Kai Hong ; Christian G. Kohler ; Mary E. March ; Amber A. Parker ; Ani Nenkova
Abstract: We present a system for automatic identification of schizophrenic patients and healthy controls based on narratives the subjects recounted about emotional experiences in their own life. The focus of the study is to identify the lexical features that distinguish the two populations. We report the results of feature selection experiments that demonstrate that the classifier can achieve accuracy on patient level prediction as high as 76.9% with only a small set of features. We provide an in-depth discussion of the lexical features that distinguish the two groups and the unexpected relationship between emotion types of the narratives and the accuracy of patient status prediction.
4 0.83339548 78 emnlp-2012-Learning Lexicon Models from Search Logs for Query Expansion
Author: Jianfeng Gao ; Shasha Xie ; Xiaodong He ; Alnur Ali
Abstract: This paper explores log-based query expansion (QE) models for Web search. Three lexicon models are proposed to bridge the lexical gap between Web documents and user queries. These models are trained on pairs of user queries and titles of clicked documents. Evaluations on a real world data set show that the lexicon models, integrated into a ranker-based QE system, not only significantly improve the document retrieval performance but also outperform two state-of-the-art log-based QE methods.
5 0.49589935 5 emnlp-2012-A Discriminative Model for Query Spelling Correction with Latent Structural SVM
Author: Huizhong Duan ; Yanen Li ; ChengXiang Zhai ; Dan Roth
Abstract: Discriminative training in query spelling correction is difficult due to the complex internal structures of the data. Recent work on query spelling correction suggests a two stage approach a noisy channel model that is used to retrieve a number of candidate corrections, followed by discriminatively trained ranker applied to these candidates. The ranker, however, suffers from the fact the low recall of the first, suboptimal, search stage. This paper proposes to directly optimize the search stage with a discriminative model based on latent structural SVM. In this model, we treat query spelling correction as a multiclass classification problem with structured input and output. The latent structural information is used to model the alignment of words in the spelling correction process. Experiment results show that as a standalone speller, our model outperforms all the baseline systems. It also attains a higher recall compared with the noisy channel model, and can therefore serve as a better filtering stage when combined with a ranker.
6 0.48439312 14 emnlp-2012-A Weakly Supervised Model for Sentence-Level Semantic Orientation Analysis with Multiple Experts
7 0.45735663 20 emnlp-2012-Answering Opinion Questions on Products by Exploiting Hierarchical Organization of Consumer Reviews
8 0.4437933 47 emnlp-2012-Explore Person Specific Evidence in Web Person Name Disambiguation
9 0.44058967 97 emnlp-2012-Natural Language Questions for the Web of Data
10 0.43684554 52 emnlp-2012-Fast Large-Scale Approximate Graph Construction for NLP
11 0.43344039 101 emnlp-2012-Opinion Target Extraction Using Word-Based Translation Model
12 0.43229061 137 emnlp-2012-Why Question Answering using Sentiment Analysis and Word Classes
13 0.42850947 8 emnlp-2012-A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes
14 0.41944644 120 emnlp-2012-Streaming Analysis of Discourse Participants
15 0.41443747 50 emnlp-2012-Extending Machine Translation Evaluation Metrics with Lexical Cohesion to Document Level
16 0.41203278 110 emnlp-2012-Reading The Web with Learned Syntactic-Semantic Inference Rules
17 0.408409 33 emnlp-2012-Discovering Diverse and Salient Threads in Document Collections
18 0.40699184 3 emnlp-2012-A Coherence Model Based on Syntactic Patterns
19 0.403528 63 emnlp-2012-Identifying Event-related Bursts via Social Media Activities
20 0.39622515 107 emnlp-2012-Polarity Inducing Latent Semantic Analysis