nips nips2001 nips2001-107 nips2001-107-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: David M. Blei, Andrew Y. Ng, Michael I. Jordan
Abstract: We propose a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams [6], and Hofmann's aspect model , also known as probabilistic latent semantic indexing (pLSI) [3]. In the context of text modeling, our model posits that each document is generated as a mixture of topics, where the continuous-valued mixture proportions are distributed as a latent Dirichlet random variable. Inference and learning are carried out efficiently via variational algorithms. We present empirical results on applications of this model to problems in text modeling, collaborative filtering, and text classification. 1
[1] D. Cohn and T. Hofmann. The missing link- A probabilistic model of document content and hypertext connectivity. In Advances in Neural Information Processing Systems 13, 2001.
[2] P.J. Green and S. Richardson. Modelling heterogeneity with and without the Dirichlet process. Technical Report, University of Bristol, 1998.
[3] T. Hofmann. Probabilistic latent semantic indexing. Proceedings of th e Twenty-Second Annual International SIGIR Conference, 1999.
[4] T. J. Jiang, J. B. Kadane, and J. M. Dickey. Computation of Carlson's multiple hypergeometric functions r for Bayesian applications. Journal of Computational and Graphical Statistics, 1:231- 251 , 1992.
[5] M. I. Jordan , Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. Introduction to variational methods for graphical models. Machine Learning, 37:183- 233, 1999.
[6] K. Nigam, A. Mccallum, S. Thrun, and T. Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2/3):103- 134, 2000.
[7] A. Popescul, L. H. Ungar, D. M. Pennock, and S. Lawrence. Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In Uncertainty in Artificial Intelligence, Proceedings of the Seventeenth Conference, 2001. 3These remarks also distinguish our model from the Bayesian Dirichlet/Multinomial allocation model (DMA)of [2], which is a finite alternative to the Dirichlet process . The DMA places a mixture of Dirichlet priors on p(wl z ) and sets O i = 00 for all i .