nips nips2012 nips2012-124 nips2012-124-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Michael Paul, Mark Dredze
Abstract: Latent variable models can be enriched with a multi-dimensional structure to consider the many latent factors in a text corpus, such as topic, author perspective and sentiment. We introduce factorial LDA, a multi-dimensional model in which a document is influenced by K different factors, and each word token depends on a K-dimensional vector of latent variables. Our model incorporates structured word priors and learns a sparse product of factors. Experiments on research abstracts show that our model can learn latent factors such as research topic, scientific discipline, and focus (methods vs. applications). Our modeling improvements reduce test perplexity and improve human interpretability of the discovered factors. 1
[1] D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. JMLR, 2003.
[2] Q. Mei, X. Ling, M. Wondra, H. Su, and C. Zhai. Topic sentiment mixture: modeling facets and opinions in weblogs. In WWW, 2007.
[3] M. Paul and R. Girju. A two-dimensional topic-aspect model for discovering multi-faceted topics. In AAAI, 2010.
[4] J. Eisenstein, A. Ahmed, and E. P. Xing. Sparse additive generative models of text. In ICML, 2011.
[5] W. Y. Wang, E. Mayfield, S. Naidu, and J. Dittmar. Historical analysis of legal opinions with a sparse mixed-effects latent variable model. In ACL, pages 740–749, July 2012.
[6] J. Chang, J. Boyd-Graber, S. Gerrish, C. Wang, and D. Blei. Reading tea leaves: How humans interpret topic models. In NIPS, 2009.
[7] D. Mimno and A. McCallum. Topic models conditioned on arbitrary features with dirichlet-multinomial regression. In UAI, 2008.
[8] T. Griffiths and Z. Ghahramani. Infinite latent feature models and the Indian buffet process. In NIPS, 2006.
[9] S. Williamson, C. Wang, K. Heller, and D. Blei. The IBP-compound dirichlet process and its application to focused topic modeling. In ICML, 2010.
[10] A. Ahmed and E. P. Xing. Staying informed: supervised and semi-supervised multi-view topical analysis of ideological perspective. In EMNLP, pages 1140–1150, 2010.
[11] M. Paul and R. Girju. Cross-cultural analysis of blogs and forums with mixed-collection topic models. In EMNLP, pages 1408–1417, August 2009.
[12] D. Zhang, C. Zhai, J. Han, A. Srivastava, and N. Oza. Topic modeling for OLAP on multidimensional text databases: topic cube and its applications. Statistical Analysis and Data Mining, 2, 2009.
[13] T. Hofmann. Probabilistic latent semantic indexing. In SIGIR, 1999.
[14] S. Dasgupta and V. Ng. Mining clustering dimensions. In ICML, 2010.
[15] I. Porteous, E. Bart, and M. Welling. Multi-HDP: a non parametric Bayesian model for tensor factorization. In AAAI, pages 1487–1490, 2008.
[16] L. Mackey, D. Weiss, and M. I. Jordan. Mixed membership matrix factorization. In ICML, 2010.
[17] G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Comput., 14:1771–1800, August 2002.
[18] J. Boyd-Graber and D. Blei. Syntactic topic models. In NIPS, 2008.
[19] M. R. Gormley, M. Dredze, B. Van Durme, and J. Eisner. Shared components topic models. In NAACL, 2010.
[20] C. Wang and D. Blei. Decoupling sparsity and smoothness in the discrete hierarchical Dirichlet process. In NIPS, 2009.
[21] L. Meier, S. van de Geer, and P. B¨ hlmann. The group lasso for logistic regression. Journal Of The Royal u Statistical Society Series B, 70(1):53–71, 2008.
[22] H. Wallach, D. Mimno, and A. McCallum. Rethinking LDA: Why priors matter. In NIPS, 2009.
[23] T. Griffiths and M. Steyvers. Finding scientific topics. In Proceedings of the National Academy of Sciences of the United States of America, 2004.
[24] M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth. The author-topic model for authors and documents. In UAI, 2004.
[25] Michael J. Paul. Mixed membership Markov models for unsupervised conversation modeling. In EMNLPCoNLL, 2012. 9