nips nips2012 nips2012-154 knowledge-graph by maker-knowledge-mining

154 nips-2012-How They Vote: Issue-Adjusted Models of Legislative Behavior

Source: pdf

Author: Sean Gerrish, David M. Blei

Abstract: We develop a probabilistic model of legislative data that uses the text of the bills to uncover lawmakers’ positions on speciﬁc political issues. Our model can be used to explore how a lawmaker’s voting patterns deviate from what is expected and how that deviation depends on what is being voted on. We derive approximate posterior inference algorithms based on variational methods. Across 12 years of legislative data, we demonstrate both improvement in heldout predictive performance and the model’s utility in interpreting an inherently multi-dimensional space. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract We develop a probabilistic model of legislative data that uses the text of the bills to uncover lawmakers’ positions on speciﬁc political issues. [sent-7, score-0.642]

2 1 Introduction Legislative behavior centers around the votes made by lawmakers. [sent-11, score-0.333]

3 Capturing regularity in these votes, and characterizing patterns of legislative behavior, is one of the main goals of quantitative political science. [sent-12, score-0.328]

4 Voting behavior exhibits enough regularity that simple statistical models, particularly ideal point models, easily capture the broad political structure of legislative bodies. [sent-13, score-0.78]

5 However, some lawmakers do not ﬁt neatly into the assumptions made by these models. [sent-14, score-0.464]

6 In this paper, we develop a new model of legislative behavior that captures when and how lawmakers vote differently than expected. [sent-15, score-0.762]

7 Ideal point models assume that lawmakers and bills are represented as points in a latent space. [sent-16, score-0.823]

8 Given the data of how each lawmaker votes on each bill (known as a roll call), we can use ideal point models to infer the latent position of each lawmaker. [sent-18, score-1.281]

9 politics, these inferred positions reveal the commonly-known political spectrum: right-wing lawmakers are at one extreme, and left-wing lawmakers are at the other. [sent-21, score-1.056]

10 Figure 1 illustrates example inferences from an ideal point model. [sent-22, score-0.424]

11 But there are some votes that ideal point models fail to capture. [sent-23, score-0.729]

12 For example, Ronald Paul, Republican representative from Texas, and Dennis Kucinich, Democratic representative from Ohio, are poorly modeled by ideal points because they diverge from the left-right spectrum on issues like foreign policy. [sent-24, score-0.547]

13 Because some lawmakers deviate from their party on certain issues, their positions on these issues are not captured by ideal point models. [sent-25, score-1.044]

14 To this end, we develop the issue-adjusted ideal point model, a latent variable model of roll-call data that accounts for the contents of the bills that lawmakers are voting on. [sent-26, score-1.291]

15 The votes on a bill depend on a lawmaker’s position, adjusted for the bill’s content. [sent-28, score-0.554]

16 The text of the bill encodes the issues it discusses. [sent-29, score-0.365]

17 1 −2 −1 0 1 2 3 Figure 1: Traditional ideal points separate Republicans (red) from Democrats (blue). [sent-31, score-0.409]

18 exceptional voting patterns of individual legislators, and it provides a richer description of lawmakers’ voting behavior than the models traditionally used in political science. [sent-32, score-0.415]

19 We show that our model gives a better ﬁt to legislative data and provides an interesting exploratory tool for analyzing legislative behavior. [sent-35, score-0.426]

20 2 Exceptional issue voting We ﬁrst review ideal point models of legislative roll call data and discuss their limitations. [sent-36, score-0.898]

21 Applied to voting records, one-dimensional ideal point models place lawmakers on an interpretable political spectrum. [sent-40, score-1.129]

22 One-dimensional ideal point models posit an ideal point xu ∈ R for each lawmaker u. [sent-42, score-1.151]

23 Each bill d is characterized by its polarity ad and its popularity bd . [sent-43, score-0.531]

24 1 The probability that lawmaker u votes “Yes” on bill d is given by the logistic regression p(vud = yes | xu , ad , bd ) = σ(xu ad + bd ), (1) exp(s) where σ(s) = 1+exp(s) is the logistic function. [sent-44, score-1.329]

25 2 When the popularity of a bill bd is high, nearly everyone votes “Yes”; when the popularity is low, nearly everyone votes “No”. [sent-45, score-1.027]

26 When the popularity is near zero, the probability that a lawmaker votes “Yes” depends on how her ideal point xu interacts with bill polarity ad . [sent-46, score-1.457]

27 Given a matrix of votes, we can infer the ideal point of each lawmaker. [sent-48, score-0.424]

28 The model has clearly separated lawmakers by their political party (colour) and provides an intuitive measure of their political leanings. [sent-52, score-0.761]

29 The ideal point model assumes that lawmakers are ordered. [sent-59, score-0.888]

30 Lawmakers to one side of the cut point are more likely to support the bill, and lawmakers to d the other side are likely to reject it. [sent-61, score-0.506]

31 For lawmakers like Paul and Hill, this assumption is too strong because their voting behavior does not ﬁt neatly into a single ordering. [sent-62, score-0.605]

32 Their location among the other lawmakers changes with different bills. [sent-63, score-0.464]

33 Paul consistently votes against United States involvement in foreign military engagements, a position that contrasts with other Republicans. [sent-67, score-0.384]

34 An issue is any federal policy area, such as “ﬁnancial regulation,” “foreign policy,” “civil liberties,” or “education,” on which lawmakers are expected to take positions. [sent-69, score-0.57]

35 Many ideal point models use a probit function instead [1, 3]. [sent-73, score-0.424]

36 In the issue-adjusted ideal point model, lawmakers’ ideal points change when they vote on certain issues, such as Taxation. [sent-75, score-0.903]

37 Right: the issue-adjusted ideal point model, which models votes vud from lawmakers and legislative items. [sent-77, score-1.484]

38 Classic item response theory models votes v using xu and ad , bd . [sent-78, score-0.633]

39 The model we will introduce uses lawmakers’ votes and the text of bills to model deviations like this, on a variety of issues. [sent-82, score-0.619]

40 We now describe the issue-adjusted ideal point model, a new model of lawmaker behavior that takes into account both the content of the bills and the voting patterns of the lawmakers. [sent-85, score-1.07]

41 We build on the ideal point model so that each lawmaker’s ideal point can be adjusted for each issue. [sent-86, score-0.848]

42 ) In our proposed model, each lawmaker is also associated with a K-vector zu ∈ RK , which describes how her ideal point changes for bills about each issue. [sent-90, score-1.049]

43 We use these variables in a model based on the traditional ideal point model of Equation 1. [sent-91, score-0.45]

44 As above, xu is the ideal point for lawmaker u and ad , bd are the polarity and popularity of bill d. [sent-92, score-1.258]

45 In our model, votes are modeled with a logistic regression p(vud |ad , bd , zu , xu , wd ) = σ (zu Eq [θd |wd ] + xu )ad + bd , (2) where we use an estimate Eq [θd |wd ] of the bill’s issue vector from its words wd as described below. [sent-93, score-1.041]

46 We put standard normal priors on the ideal points, polarity, and difﬁculty variables. [sent-94, score-0.382]

47 With a classic ideal point model, a lawmaker u’s ideal point, xu , gives his position on each issue, including Finance. [sent-100, score-1.172]

48 With the issue-adjusted ideal point model, his effective ideal point for Finance, xu + zu,Finance , gives his position on Finance. [sent-101, score-0.969]

49 When zu,k = 0 for all u, k, the model becomes the classic ideal point model. [sent-103, score-0.454]

50 Given a collection of votes and a coding of bills to issues, posterior estimates of the ideal points and per-issue adjustments give us a window into voting behavior that is not available to classic ideal point models. [sent-105, score-1.716]

51 Equation 2 adjusts a lawmaker’s ideal point by using the conditional expectation of a bill’s thematic labels θd given its words wd . [sent-107, score-0.485]

52 Labeled LDA is a topic model, a bag-of-words model that assumes a set of themes for the collection of bills and that each bill exhibits a mixture of those themes. [sent-109, score-0.598]

53 These subject codes describe the bills using phrases which correspond to traditional issues, such as Civil rights and National security. [sent-116, score-0.361]

54 In the issueadjusted ideal point model (Equation 2), E [θd ] was treated as observed when estimating the posterior distribution p(xu , ad , bd , zd |E [θd |wd ] , vud ). [sent-122, score-0.779]

55 Some political scientists have used higher-dimensional ideal points, where each legislator is attached to a vector of ideal points xu ∈ RK and each bill polarization ad takes the same dimension K [11]. [sent-127, score-1.364]

56 The probability of a lawmaker voting “Yes” is σ(xT ad + bd ). [sent-128, score-0.542]

57 The principal component of u ideal points explains most of the variance and explains party afﬁliation. [sent-129, score-0.45]

58 Congressional ﬂoor debates to predict whether speeches support or oppose pending legislation [12] and predicting whether a bill will survive congressional committee by incorporating a number of features, including bill text [13]. [sent-134, score-0.647]

59 Gerrish and Blei aimed to predict votes on bills which had not yet received any votes [14]. [sent-136, score-0.9]

60 Their model ﬁts ad and bd using supervised topics, but the underlying voting model was one-dimensional: it could not model individual votes better than a one-dimensional ideal point model. [sent-137, score-1.056]

61 created a Bayesian nonparametric model of votes and text over time [15]. [sent-139, score-0.329]

62 4 4 3 Posterior estimation The central computational challenge in this model is to uncover lawmakers’ issue preferences zu by using the their votes v and bills’ issues θd . [sent-149, score-0.623]

63 Bayesian ideal point models are usually ﬁt with Gibbs sampling [2, 3, 5, 18]. [sent-151, score-0.424]

64 The similarity between ideal points ﬁt with variational inference and MCMC has been demonstrated in Gerrish in Blei [14]. [sent-159, score-0.449]

65 We compare the posterior ﬁt with this model to the same data ﬁt with traditional ideal points and validate the model quantitatively. [sent-170, score-0.462]

66 For each congress, we considered only bills for which votes were explicitly recorded in a roll-call. [sent-187, score-0.595]

67 We ignored votes on bills for which text was unavailable. [sent-188, score-0.619]

68 5 Table 1: Average log-likelihood of heldout votes using six-fold cross validation. [sent-196, score-0.366]

69 The issue-adjusted model yields higher heldout log-likelihood for all congresses in both chambers than a standard ideal point model. [sent-198, score-0.553]

70 187 Comparison of classic and exploratory ideal points How do classic ideal points compare with issue-adjusted ideal points? [sent-242, score-1.286]

71 We ﬁt classic ideal points to the 111th House (2009 to 2010) to compare them with issue-adjusted ideal points xu from the ˜ same period, using regularization λ = 1. [sent-243, score-0.936]

72 The models’ ideal points xu were very similar, correlated ˜ at 0. [sent-244, score-0.497]

73 While traditional ideal points cleanly separate Democrats and Republicans in this period, issue-adjusted ideal points provide an even cleaner break between the parties. [sent-246, score-0.844]

74 For each fold, we computed the average predictive log-likelihood log p(vudTest |vudTrain ) = log p(vudTest |˜u , zu , ad , ˜d , Eq [θd |w]) of the test x ˜ ˜ b votes and averaged this across folds. [sent-253, score-0.533]

75 We compared these with the ideal point model, evaluating the latter in the same way. [sent-254, score-0.424]

76 Note that we cannot evaluate how well this model predicts votes on a heldout bill d. [sent-257, score-0.615]

77 As with the ideal point model, our model cannot predict ad , ˜d without votes on d. [sent-258, score-0.837]

78 We compared the issue-adjusted model’s ability to represent heldout votes with the ideal point model. [sent-262, score-0.79]

79 For comparison we also ﬁt an ideal point model to each of these congresses. [sent-264, score-0.424]

80 In all Congresses and both chambers, the issue-adjusted model represents heldout votes with higher log-likelihood than an ideal point model. [sent-265, score-0.79]

81 We used a permutation test to understand how the issue-adjusted model improves upon ideal point models. [sent-276, score-0.424]

82 4 Analyzing issues, lawmakers, and bills In this section we take a closer look at how issue adjustments improve on ideal points and demonstrate how the issue-adjusted ideal point model can be used to analyze speciﬁc lawmakers. [sent-296, score-1.319]

83 We can measure the improvement by comparing the training likelihoods of votes in the issue-adjusted and traditional ideal point models. [sent-298, score-0.787]

84 The training log-likelihood of each vote is Jud = 1{vud =Yes } p − log(1 + exp(p)), (6) T ˜d is the log-odds of a vote under the issue adjusted voting where p = (˜u + zu Eq [θd |w])˜d + b x ˜ a model. [sent-299, score-0.479]

85 The corresponding log-likelihood Iud under the ideal point model is p = xu ad + ˜d . [sent-300, score-0.62]

86 This is a result predicted by procedural cartel theory [23, 24, 25, 26], which posits that lawmakers will be more polarized in procedural votes (which describe how Congress will be run) than substantive votes (the issues discussed during elections). [sent-306, score-1.264]

87 Some of most-improved lawmakers were Ron Paul and Donald Young. [sent-314, score-0.464]

88 9% with issue offsets, placing him among the two most-improved lawmakers with this model. [sent-326, score-0.57]

89 Young stood out in a topic used frequently in House bills about naming local landmarks. [sent-333, score-0.326]

90 Young voted against the majority of his party (and the House in general) on a series of largely symbolic bills and resolutions. [sent-334, score-0.381]

91 ” Young’s divergent voting was also evident in a series of votes against naming various landmarks–such as post ofﬁces–in a topic about such symbolic votes. [sent-336, score-0.454]

92 Notice that Young’s ideal point is not particularly distinctive: using the ideal point alone, we would not recognize his unique voting behavior. [sent-337, score-0.961]

93 The bill which decreased the most from the ideal point model in the 111th House was the Consolidated Land, Energy, and Aquatic Resources Act of 2010 (H. [sent-341, score-0.673]

94 This bill had substantial weight in ﬁve issues, with most in Public lands and natural resources, Energy, and Land transfers, but its placement in many issues harmed our predictions. [sent-344, score-0.364]

95 This effect—worse performance on bills about many issues—suggests that methods which represent bills more sparsely may perform better than the current model. [sent-345, score-0.58]

96 5 Discussion Traditional models of roll call data cannot capture how individual lawmakers deviate from their latent position on the political spectrum. [sent-346, score-0.703]

97 In this paper, we developed a model that captures how lawmakers vary, issue by issue, and used the text of the bills to attach speciﬁc votes to speciﬁc issues. [sent-347, score-1.189]

98 We demonstrated, across 12 years of legislative data, that this model better captures lawmaker behavior. [sent-348, score-0.415]

99 For example, lawmakers make some (but not all) issue positions public. [sent-351, score-0.57]

100 Dynamic ideal point estimation via Markov chain Monte Carlo for the U. [sent-375, score-0.424]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('lawmakers', 0.464), ('ideal', 0.382), ('votes', 0.305), ('bills', 0.29), ('bill', 0.249), ('lawmaker', 0.215), ('legislative', 0.2), ('house', 0.158), ('political', 0.128), ('congressional', 0.125), ('zu', 0.12), ('voting', 0.113), ('ad', 0.108), ('issue', 0.106), ('bd', 0.106), ('issues', 0.092), ('vud', 0.091), ('adjustments', 0.09), ('xu', 0.088), ('congress', 0.074), ('vote', 0.07), ('congresses', 0.068), ('paul', 0.061), ('wd', 0.061), ('heldout', 0.061), ('zuk', 0.06), ('young', 0.058), ('republicans', 0.057), ('senate', 0.057), ('roll', 0.055), ('voted', 0.05), ('eq', 0.049), ('foreign', 0.046), ('democrats', 0.045), ('ethnic', 0.045), ('impk', 0.045), ('iud', 0.045), ('jud', 0.045), ('racial', 0.045), ('rights', 0.045), ('yes', 0.044), ('gerrish', 0.043), ('point', 0.042), ('party', 0.041), ('guard', 0.04), ('variational', 0.04), ('finance', 0.038), ('lda', 0.038), ('blei', 0.037), ('polarity', 0.037), ('topic', 0.036), ('ro', 0.035), ('aul', 0.034), ('procedural', 0.034), ('exceptional', 0.033), ('land', 0.033), ('position', 0.033), ('health', 0.033), ('improvement', 0.032), ('donald', 0.032), ('popularity', 0.031), ('legislators', 0.03), ('substantive', 0.03), ('classic', 0.03), ('offsets', 0.029), ('period', 0.029), ('permuted', 0.028), ('politics', 0.028), ('republican', 0.028), ('behavior', 0.028), ('topics', 0.027), ('posterior', 0.027), ('points', 0.027), ('exploratory', 0.026), ('item', 0.026), ('sessions', 0.026), ('keith', 0.026), ('gary', 0.026), ('traditional', 0.026), ('labeled', 0.026), ('ym', 0.025), ('david', 0.025), ('months', 0.025), ('ron', 0.025), ('text', 0.024), ('james', 0.023), ('deviate', 0.023), ('bush', 0.023), ('democratic', 0.023), ('ell', 0.023), ('enelow', 0.023), ('issueadjusted', 0.023), ('jes', 0.023), ('lands', 0.023), ('mathew', 0.023), ('mcc', 0.023), ('ramage', 0.023), ('themes', 0.023), ('vudtest', 0.023), ('ronald', 0.021)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000001 154 nips-2012-How They Vote: Issue-Adjusted Models of Legislative Behavior

Author: Sean Gerrish, David M. Blei

2 0.21180925 166 nips-2012-Joint Modeling of a Matrix with Associated Text via Latent Binary Features

Author: Xianxing Zhang, Lawrence Carin

Abstract: A new methodology is developed for joint analysis of a matrix and accompanying documents, with the documents associated with the matrix rows/columns. The documents are modeled with a focused topic model, inferring interpretable latent binary features for each document. A new matrix decomposition is developed, with latent binary features associated with the rows/columns, and with imposition of a low-rank constraint. The matrix decomposition and topic model are coupled by sharing the latent binary feature vectors associated with each. The model is applied to roll-call data, with the associated documents deﬁned by the legislation. Advantages of the proposed model are demonstrated for prediction of votes on a new piece of legislation, based only on the observed text of legislation. The coupling of the text and legislation is also shown to yield insight into the properties of the matrix decomposition for roll-call data. 1

3 0.069174871 52 nips-2012-Bayesian Nonparametric Modeling of Suicide Attempts

Author: Francisco Ruiz, Isabel Valera, Carlos Blanco, Fernando Pérez-Cruz

Abstract: The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) database contains a large amount of information, regarding the way of life, medical conditions, etc., of a representative sample of the U.S. population. In this paper, we are interested in seeking the hidden causes behind the suicide attempts, for which we propose to model the subjects using a nonparametric latent model based on the Indian Buffet Process (IBP). Due to the nature of the data, we need to adapt the observation model for discrete random variables. We propose a generative model in which the observations are drawn from a multinomial-logit distribution given the IBP matrix. The implementation of an efﬁcient Gibbs sampler is accomplished using the Laplace approximation, which allows integrating out the weighting factors of the multinomial-logit likelihood model. Finally, the experiments over the NESARC database show that our model properly captures some of the hidden causes that model suicide attempts. 1

4 0.057489943 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

Author: Michael Bryant, Erik B. Sudderth

Abstract: Variational methods provide a computationally scalable alternative to Monte Carlo methods for large-scale, Bayesian nonparametric learning. In practice, however, conventional batch and online variational methods quickly become trapped in local optima. In this paper, we consider a nonparametric topic model based on the hierarchical Dirichlet process (HDP), and develop a novel online variational inference algorithm based on split-merge topic updates. We derive a simpler and faster variational approximation of the HDP, and show that by intelligently splitting and merging components of the variational posterior, we can achieve substantially better predictions of test data than conventional online and batch variational algorithms. For streaming analysis of large datasets where batch analysis is infeasible, we show that our split-merge updates better capture the nonparametric properties of the underlying model, allowing continual learning of new topics.

5 0.050778538 124 nips-2012-Factorial LDA: Sparse Multi-Dimensional Text Models

Author: Michael Paul, Mark Dredze

Abstract: Latent variable models can be enriched with a multi-dimensional structure to consider the many latent factors in a text corpus, such as topic, author perspective and sentiment. We introduce factorial LDA, a multi-dimensional model in which a document is inﬂuenced by K different factors, and each word token depends on a K-dimensional vector of latent variables. Our model incorporates structured word priors and learns a sparse product of factors. Experiments on research abstracts show that our model can learn latent factors such as research topic, scientiﬁc discipline, and focus (methods vs. applications). Our modeling improvements reduce test perplexity and improve human interpretability of the discovered factors. 1

6 0.048944727 355 nips-2012-Truncation-free Online Variational Inference for Bayesian Nonparametric Models

7 0.045602828 165 nips-2012-Iterative ranking from pair-wise comparisons

8 0.042349212 136 nips-2012-Forward-Backward Activation Algorithm for Hierarchical Hidden Markov Models

9 0.037919629 303 nips-2012-Searching for objects driven by context

10 0.037588749 274 nips-2012-Priors for Diversity in Generative Latent Variable Models

11 0.03736661 168 nips-2012-Kernel Latent SVM for Visual Recognition

12 0.036870554 286 nips-2012-Random Utility Theory for Social Choice

13 0.036674958 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

14 0.036277965 220 nips-2012-Monte Carlo Methods for Maximum Margin Supervised Topic Models

15 0.036191441 19 nips-2012-A Spectral Algorithm for Latent Dirichlet Allocation

16 0.034717109 105 nips-2012-Dynamic Pruning of Factor Graphs for Maximum Marginal Prediction

17 0.031443309 129 nips-2012-Fast Variational Inference in the Conjugate Exponential Family

18 0.030919407 133 nips-2012-Finding Exemplars from Pairwise Dissimilarities via Simultaneous Sparse Recovery

19 0.030006634 316 nips-2012-Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models

20 0.029048854 298 nips-2012-Scalable Inference of Overlapping Communities

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.086), (1, 0.014), (2, -0.022), (3, -0.008), (4, -0.087), (5, -0.03), (6, -0.01), (7, 0.02), (8, 0.041), (9, -0.021), (10, 0.051), (11, 0.076), (12, 0.005), (13, -0.011), (14, 0.01), (15, 0.055), (16, 0.017), (17, 0.027), (18, -0.018), (19, 0.045), (20, -0.014), (21, 0.004), (22, 0.018), (23, -0.034), (24, 0.007), (25, 0.01), (26, 0.044), (27, 0.059), (28, -0.033), (29, -0.011), (30, 0.034), (31, 0.023), (32, -0.016), (33, -0.023), (34, -0.01), (35, -0.051), (36, -0.048), (37, 0.074), (38, -0.043), (39, -0.01), (40, -0.034), (41, 0.002), (42, -0.003), (43, 0.001), (44, 0.122), (45, 0.124), (46, 0.044), (47, -0.025), (48, 0.089), (49, 0.057)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.9029904 154 nips-2012-How They Vote: Issue-Adjusted Models of Legislative Behavior

Author: Sean Gerrish, David M. Blei

2 0.71811843 166 nips-2012-Joint Modeling of a Matrix with Associated Text via Latent Binary Features

Author: Xianxing Zhang, Lawrence Carin

3 0.61035603 345 nips-2012-Topic-Partitioned Multinetwork Embeddings

Author: Peter Krafft, Juston Moore, Bruce Desmarais, Hanna M. Wallach

Abstract: We introduce a new Bayesian admixture model intended for exploratory analysis of communication networks—speciﬁcally, the discovery and visualization of topic-speciﬁc subnetworks in email data sets. Our model produces principled visualizations of email networks, i.e., visualizations that have precise mathematical interpretations in terms of our model and its relationship to the observed data. We validate our modeling assumptions by demonstrating that our model achieves better link prediction performance than three state-of-the-art network models and exhibits topic coherence comparable to that of latent Dirichlet allocation. We showcase our model’s ability to discover and visualize topic-speciﬁc communication patterns using a new email data set: the New Hanover County email network. We provide an extensive analysis of these communication patterns, leading us to recommend our model for any exploratory analysis of email networks or other similarly-structured communication data. Finally, we advocate for principled visualization as a primary objective in the development of new network models. 1

4 0.5666306 220 nips-2012-Monte Carlo Methods for Maximum Margin Supervised Topic Models

Author: Qixia Jiang, Jun Zhu, Maosong Sun, Eric P. Xing

Abstract: An effective strategy to exploit the supervising side information for discovering predictive topic representations is to impose discriminative constraints induced by such information on the posterior distributions under a topic model. This strategy has been adopted by a number of supervised topic models, such as MedLDA, which employs max-margin posterior constraints. However, unlike the likelihoodbased supervised topic models, of which posterior inference can be carried out using the Bayes’ rule, the max-margin posterior constraints have made Monte Carlo methods infeasible or at least not directly applicable, thereby limited the choice of inference algorithms to be based on variational approximation with strict mean ﬁeld assumptions. In this paper, we develop two efﬁcient Monte Carlo methods under much weaker assumptions for max-margin supervised topic models based on an importance sampler and a collapsed Gibbs sampler, respectively, in a convex dual formulation. We report thorough experimental results that compare our approach favorably against existing alternatives in both accuracy and efﬁciency.

5 0.53513992 52 nips-2012-Bayesian Nonparametric Modeling of Suicide Attempts

Author: Francisco Ruiz, Isabel Valera, Carlos Blanco, Fernando Pérez-Cruz

6 0.51568514 332 nips-2012-Symmetric Correspondence Topic Models for Multilingual Text Analysis

7 0.50299001 124 nips-2012-Factorial LDA: Sparse Multi-Dimensional Text Models

8 0.47581115 12 nips-2012-A Neural Autoregressive Topic Model

9 0.44916698 274 nips-2012-Priors for Diversity in Generative Latent Variable Models

10 0.43297586 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

11 0.42993158 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

12 0.42342949 19 nips-2012-A Spectral Algorithm for Latent Dirichlet Allocation

13 0.41785988 36 nips-2012-Adaptive Stratified Sampling for Monte-Carlo integration of Differentiable functions

14 0.40271375 47 nips-2012-Augment-and-Conquer Negative Binomial Processes

15 0.40235135 246 nips-2012-Nonparametric Max-Margin Matrix Factorization for Collaborative Prediction

16 0.37665009 355 nips-2012-Truncation-free Online Variational Inference for Bayesian Nonparametric Models

17 0.3713519 137 nips-2012-From Deformations to Parts: Motion-based Segmentation of 3D Objects

18 0.35697088 192 nips-2012-Learning the Dependency Structure of Latent Factors

19 0.3513737 184 nips-2012-Learning Probability Measures with respect to Optimal Transport Metrics

20 0.33326238 125 nips-2012-Factoring nonnegative matrices with linear programs

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.05), (17, 0.011), (21, 0.029), (31, 0.015), (38, 0.065), (39, 0.08), (42, 0.032), (54, 0.015), (55, 0.033), (74, 0.036), (76, 0.074), (80, 0.07), (92, 0.027), (98, 0.348)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.73537827 154 nips-2012-How They Vote: Issue-Adjusted Models of Legislative Behavior

Author: Sean Gerrish, David M. Blei

2 0.61378014 267 nips-2012-Perceptron Learning of SAT

Author: Alex Flint, Matthew Blaschko

Abstract: Boolean satisﬁability (SAT) as a canonical NP-complete decision problem is one of the most important problems in computer science. In practice, real-world SAT sentences are drawn from a distribution that may result in efﬁcient algorithms for their solution. Such SAT instances are likely to have shared characteristics and substructures. This work approaches the exploration of a family of SAT solvers as a learning problem. In particular, we relate polynomial time solvability of a SAT subset to a notion of margin between sentences mapped by a feature function into a Hilbert space. Provided this mapping is based on polynomial time computable statistics of a sentence, we show that the existance of a margin between these data points implies the existance of a polynomial time solver for that SAT subset based on the Davis-Putnam-Logemann-Loveland algorithm. Furthermore, we show that a simple perceptron-style learning rule will ﬁnd an optimal SAT solver with a bounded number of training updates. We derive a linear time computable set of features and show analytically that margins exist for important polynomial special cases of SAT. Empirical results show an order of magnitude improvement over a state-of-the-art SAT solver on a hardware veriﬁcation task. 1

3 0.56390268 55 nips-2012-Bayesian Warped Gaussian Processes

Author: Miguel Lázaro-gredilla

Abstract: Warped Gaussian processes (WGP) [1] model output observations in regression tasks as a parametric nonlinear transformation of a Gaussian process (GP). The use of this nonlinear transformation, which is included as part of the probabilistic model, was shown to enhance performance by providing a better prior model on several data sets. In order to learn its parameters, maximum likelihood was used. In this work we show that it is possible to use a non-parametric nonlinear transformation in WGP and variationally integrate it out. The resulting Bayesian WGP is then able to work in scenarios in which the maximum likelihood WGP failed: Low data regime, data with censored values, classiﬁcation, etc. We demonstrate the superior performance of Bayesian warped GPs on several real data sets.

4 0.53982288 356 nips-2012-Unsupervised Structure Discovery for Semantic Analysis of Audio

Author: Sourish Chaudhuri, Bhiksha Raj

Abstract: Approaches to audio classiﬁcation and retrieval tasks largely rely on detectionbased discriminative models. We submit that such models make a simplistic assumption in mapping acoustics directly to semantics, whereas the actual process is likely more complex. We present a generative model that maps acoustics in a hierarchical manner to increasingly higher-level semantics. Our model has two layers with the ﬁrst layer modeling generalized sound units with no clear semantic associations, while the second layer models local patterns over these sound units. We evaluate our model on a large-scale retrieval task from TRECVID 2011, and report signiﬁcant improvements over standard baselines. 1

5 0.47043934 174 nips-2012-Learning Halfspaces with the Zero-One Loss: Time-Accuracy Tradeoffs

Author: Aharon Birnbaum, Shai S. Shwartz

Abstract: Given α, ϵ, we study the time complexity required to improperly learn a halfspace with misclassiﬁcation error rate of at most (1 + α) L∗ + ϵ, where L∗ is the γ γ optimal γ-margin error rate. For α = 1/γ, polynomial time and sample complexity is achievable using the hinge-loss. For α = 0, Shalev-Shwartz et al. [2011] showed that poly(1/γ) time is impossible, while learning is possible in ˜ time exp(O(1/γ)). An immediate question, which this paper tackles, is what is achievable if α ∈ (0, 1/γ). We derive positive results interpolating between the polynomial time for α = 1/γ and the exponential time for α = 0. In particular, we show that there are cases in which α = o(1/γ) but the problem is still solvable in polynomial time. Our results naturally extend to the adversarial online learning model and to the PAC learning with malicious noise model. 1

6 0.46951473 180 nips-2012-Learning Mixtures of Tree Graphical Models

7 0.40791646 248 nips-2012-Nonparanormal Belief Propagation (NPNBP)

8 0.407783 166 nips-2012-Joint Modeling of a Matrix with Associated Text via Latent Binary Features

9 0.40401816 106 nips-2012-Dynamical And-Or Graph Learning for Object Shape Modeling and Detection

10 0.38861111 249 nips-2012-Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison

11 0.38653848 351 nips-2012-Transelliptical Component Analysis

12 0.37648386 352 nips-2012-Transelliptical Graphical Models

13 0.37004873 323 nips-2012-Statistical Consistency of Ranking Methods in A Rank-Differentiable Probability Space

14 0.36703235 172 nips-2012-Latent Graphical Model Selection: Efficient Methods for Locally Tree-like Graphs

15 0.36690399 355 nips-2012-Truncation-free Online Variational Inference for Bayesian Nonparametric Models

16 0.36684608 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

17 0.36378366 197 nips-2012-Learning with Recursive Perceptual Representations

18 0.36330906 47 nips-2012-Augment-and-Conquer Negative Binomial Processes

19 0.36309847 193 nips-2012-Learning to Align from Scratch

20 0.36053517 104 nips-2012-Dual-Space Analysis of the Sparse Linear Model