nips nips2012 nips2012-89 nips2012-89-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Dahua Lin, John W. Fisher
Abstract: Mixture distributions are often used to model complex data. In this paper, we develop a new method that jointly estimates mixture models over multiple data sets by exploiting the statistical dependencies between them. Specifically, we introduce a set of latent Dirichlet processes as sources of component models (atoms), and for each data set, we construct a nonparametric mixture model by combining sub-sampled versions of the latent DPs. Each mixture model may acquire atoms from different latent DPs, while each atom may be shared by multiple mixtures. This multi-to-multi association distinguishes the proposed method from previous ones that require the model structure to be a tree or a chain, allowing more flexible designs. We also derive a sampling algorithm that jointly infers the model parameters and present experiments on both document analysis and image modeling. 1
[1] David Blei and John Lafferty. Correlated topic models. In Proc. of NIPS’06, 2006.
[2] David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993–1022, 2003.
[3] Yeonseung Chung and David B. Dunson. The local Dirichlet Process. Annals of the Inst. of Stat. Math., 63(1):59–80, 2009.
[4] Li Fei-fei. A bayesian hierarchical model for learning natural scene categories. In Proc. of CVPR’05, 2005.
[5] Amir Globerson, Gal Chechik, Fernando Pereira, and Naftali Tishby. Euclidean embedding of co-occurrence data. JMLR, 8, 2007.
[6] J. E Griffin and M. F. J Steel. Order-Based Dependent Dirichlet Processes. Journal of the American Statistical Association, 101(473):179–194, March 2006.
[7] Thomas Hofmann. Probabilistic latent semantic indexing. In Proc. of ACM SIGIR’99, 1999.
[8] Yan Ke and Rahul Sukthankar. Pca-sift: A more distinctive representation for local image descriptors. In Proc. of CVPR’04, 2004.
[9] Seyoung Kim and Padhraic Smyth. Hierarchical dirichlet processes with random effects. In Proc. of NIPS’06, 2006.
[10] Jyri J. Kivinen, Erik B. Sudderth, and Michael I. Jordan. Learning multiscale representations of natural scenes using dirichlet processes. In Proc. of CVPR’07, 2007.
[11] S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proc. of CVPR’06, 2006.
[12] Dahua Lin, Eric Grimson, and John Fisher. Construction of dependent dirichlet processes based on poisson processes. In Advances of NIPS’10, 2010.
[13] Steven N. MacEachern. Dependent Nonparametric Processes. In Proceedings of the Section on Bayesian Statistical Science, 1999.
[14] Radford M. Neal. Markov Chain Sampling Methods for Dirichlet Process Mixture Models. Journal of computational and graphical statistics, 9(2):249–265, 2000.
[15] Vinayak Rao and Yee Whye Teh. Spatial Normalized Gamma Processes. In Proc. of NIPS’09, 2009.
[16] Carl Edward Rasmussen. The Infinite Gaussian Mixture Model. In Proc. of NIPS’00, 2000.
[17] Lu Ren, David B. Dunson, and Lawrence Carin. The Dynamic Hierarchical Dirichlet Process. In Proc. of ICML’08, New York, New York, USA, 2008. ACM Press.
[18] J. Sethuraman. A Constructive Definition of Dirichlet Priors. Statistica Sinica, 4(2):639–650, 1994.
[19] Erik B. Sudderth, Antonio Torralba, William Freeman, and Alan Willsky. Describing visual scenes using transformed dirichlet processes. In Proc. of NIPS’05, 2005.
[20] Yee Whye Teh, Michael I. Jordan, Matthew J. Beal, and David M. Blei. Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101(476):1566–1581, 2006.
[21] Chang Wang, David Blei, and Fei-Fei Li. Simultaneous image classification and annotation. In Proc. of CVPR’09, 2009.
[22] J. Xiao, J. Hays, K. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Proc. of CVPR’10, 2010. 9