nips nips2012 nips2012-47 nips2012-47-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Mingyuan Zhou, Lawrence Carin
Abstract: By developing data augmentation methods unique to the negative binomial (NB) distribution, we unite seemingly disjoint count and mixture models under the NB process framework. We develop fundamental properties of the models and derive efficient Gibbs sampling inference. We show that the gamma-NB process can be reduced to the hierarchical Dirichlet process with normalization, highlighting its unique theoretical, structural and computational advantages. A variety of NB processes with distinct sharing mechanisms are constructed and applied to topic modeling, with connections to existing algorithms, showing the importance of inferring both the NB dispersion and probability parameters. 1
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35] J. F. C. Kingman. Poisson Processes. Oxford University Press, 1993. M. K. Titsias. The infinite gamma-Poisson feature model. In NIPS, 2008. R. J. Thibaux. Nonparametric Bayesian Models for Machine Learning. PhD thesis, UC Berkeley, 2008. K. T. Miller. Bayesian Nonparametric Latent Feature Models. PhD thesis, UC Berkeley, 2011. M. Zhou, L. Hannah, D. Dunson, and L. Carin. Beta-negative binomial process and Poisson factor analysis. In AISTATS, 2012. T. Broderick, L. Mackey, J. Paisley, and M. I. Jordan. Combinatorial clustering and the beta negative binomial process. arXiv:1111.1802v3, 2012. Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet processes. JASA, 2006. M. I. Jordan. Hierarchical models, nested models and completely random measures. 2010. R. L. Wolpert, M. A. Clyde, and C. Tu. Stochastic expansions using continuous dictionaries: L´ vy e Adaptive Regression Kernels. Annals of Statistics, 2011. T. S. Ferguson. A Bayesian analysis of some nonparametric problems. Ann. Statist., 1973. H. Ishwaran and M. Zarepour. Exact and approximate sum-representations for the Dirichlet process. Can. J. Statist., 2002. J. Paisley, C. Wang, and D. M. Blei. The discrete infinite logistic normal distribution. Bayesian Analysis, 2012. C. I. Bliss and R. A. Fisher. Fitting the negative binomial distribution to biological data. Biometrics, 1953. A. C. Cameron and P. K. Trivedi. Regression Analysis of Count Data. Cambridge, UK, 1998. R. Winkelmann. Econometric Analysis of Count Data. Springer, Berlin, 5th edition, 2008. M. H. Quenouille. A relation between the logarithmic, Poisson, and negative binomial series. Biometrics, 1949. N. L. Johnson, A. W. Kemp, and S. Kotz. Univariate Discrete Distributions. John Wiley & Sons, 2005. S. J. Clark and J. N. Perry. Estimation of the negative binomial parameter κ by maximum quasi-likelihood. Biometrics, 1989. M. D. Robinson and G. K. Smyth. Small-sample estimation of negative binomial dispersion, with applications to SAGE data. Biostatistics, 2008. M. Zhou, L. Li, D. Dunson, and L. Carin. Lognormal and gamma mixed negative binomial regression. In ICML, 2012. C. E. Antoniak. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist., 1974. M. D. Escobar and M. West. Bayesian density estimation and inference using mixtures. JASA, 1995. C. Wang, J. Paisley, and D. M. Blei. Online variational inference for the hierarchical Dirichlet process. In AISTATS, 2011. E. Fox, E. Sudderth, M. Jordan, and A. Willsky. Developing a tempered HDP-HMM for systems with state persistence. MIT LIDS, TR #2777, 2007. N. L. Hjort. Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist., 1990. R. Thibaux and M. I. Jordan. Hierarchical beta processes and the Indian buffet process. In AISTATS, 2007. S. Williamson, C. Wang, K. A. Heller, and D. M. Blei. The IBP compound Dirichlet process and its application to focused topic modeling. In ICML, 2010. T. L. Griffiths and Z. Ghahramani. Infinite latent feature models and the Indian buffet process. In NIPS, 2005. M. Zhou, H. Chen, J. Paisley, L. Ren, L. Li, Z. Xing, D. Dunson, G. Sapiro, and L. Carin. Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images. IEEE TIP, 2012. M. Zhou, H. Yang, G. Sapiro, D. Dunson, and L. Carin. Dependent hierarchical beta process for image interpolation and denoising. In AISTATS, 2011. L. Li, M. Zhou, G. Sapiro, and L. Carin. On the integration of topic modeling and dictionary learning. In ICML, 2011. D. Blei, A. Ng, and M. Jordan. Latent Dirichlet allocation. J. Mach. Learn. Res., 2003. D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In NIPS, 2000. J. Canny. Gap: a factor model for discrete data. In SIGIR, 2004. T. L. Griffiths and M. Steyvers. Finding scientific topics. PNAS, 2004. 9