nips nips2009 nips2009-226 nips2009-226-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Vinayak Rao, Yee W. Teh
Abstract: Dependent Dirichlet processes (DPs) are dependent sets of random measures, each being marginally DP distributed. They are used in Bayesian nonparametric models when the usual exchangeability assumption does not hold. We propose a simple and general framework to construct dependent DPs by marginalizing and normalizing a single gamma process over an extended space. The result is a set of DPs, each associated with a point in a space such that neighbouring DPs are more dependent. We describe Markov chain Monte Carlo inference involving Gibbs sampling and three different Metropolis-Hastings proposals to speed up convergence. We report an empirical study of convergence on a synthetic dataset and demonstrate an application of the model to topic modeling through time. 1
[1] T. S. Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2):209–230, 1973.
[2] M. J. Beal, Z. Ghahramani, and C. E. Rasmussen. The infinite hidden Markov model. In Advances in Neural Information Processing Systems, volume 14, 2002.
[3] Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101(476):1566–1581, 2006.
[4] C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 21, 2006.
[5] M. Johnson, T. L. Griffiths, and S. Goldwater. Adaptor grammars: A framework for specifying compositional nonparametric Bayesian models. In Advances in Neural Information Processing Systems, volume 19, 2007.
[6] S. MacEachern. Dependent nonparametric processes. In Proceedings of the Section on Bayesian Statistical Science. American Statistical Association, 1999.
[7] L. E. Nieto-Barajas, I. Pruenster, and S. G. Walker. Normalized random measures driven by increasing additive processes. Annals of Statistics, 32(6):2343–2360, 2004.
[8] L. F. James, A. Lijoi, and I. Pruenster. Bayesian inference via classes of normalized random measures. ICER Working Papers - Applied Mathematics Series 5-2005, ICER - International Centre for Economic Research, April 2005.
[9] J. F. C. Kingman. Completely random measures. Pacific Journal of Mathematics, 21(1):59–78, 1967.
[10] J. F. C. Kingman. Poisson Processes. Oxford University Press, 1993.
[11] P. M¨ ller, F. A. Quintana, and G. Rosner. A method for combining inference across related nonparametric u Bayesian models. Journal of the Royal Statistical Society, 66:735–749, 2004.
[12] S. Jain and R. M. Neal. A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model. Technical report, Department of Statistics, University of Toronto, 2004.
[13] J. A. Duan, M. Guindani, and A. E. Gelfand. Generalized spatial Dirichlet process models. Biometrika, 94(4):809–825, 2007.
[14] A. Rodr´guez, D. B. Dunson, and A. E. Gelfand. The nested Dirichlet process. Technical Report 2006-19, ı Institute of Statistics and Decision Sciences, Duke University, 2006.
[15] D. B. Dunson, Y. Xue, and L. Carin. The matrix stick-breaking process: Flexible Bayes meta analysis. Technical Report 07-03, Institute of Statistics and Decision Sciences, Duke University, 2007. http://ftp.isds.duke.edu/WorkingPapers/07-03.html.
[16] N. Srebro and S. Roweis. Time-varying topic models using dependent Dirichlet processes. Technical Report UTML-TR-2005-003, Department of Computer Science, University of Toronto, 2005.
[17] J. E. Griffin and M. F. J. Steel. Order-based dependent Dirichlet processes. Journal of the American Statistical Association, Theory and Methods, 101:179–194, 2006.
[18] J. E. Griffin. The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference. Technical report, Department of Statistics, University of Warwick, 2007.
[19] F. Caron, M. Davy, and A. Doucet. Generalized Polya urn for time-varying Dirichlet process mixtures. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, volume 23, 2007.
[20] A. Ahmed and E. P. Xing. Dynamic non-parametric mixture models and the recurrent Chinese restaurant process. In Proceedings of The Eighth SIAM International Conference on Data Mining, 2008.
[21] J. E. Griffin and M. F. J. Steel. Bayesian nonparametric modelling with the Dirichlet process regression smoother. Technical report, University of Kent and University of Warwick, 2008.
[22] J. E. Griffin and M. F. J. Steel. Generalized spatial Dirichlet process models. Technical report, University of Kent and University of Warwick, 2009.
[23] Y. Chung and D. B. Dunson. The local Dirichlet process. Annals of the Institute of Mathematical Statistics, 2009. to appear.
[24] S.N. MacEachern, A. Kottas, and A.E. Gelfand. Spatial nonparametric Bayesian models. In Proceedings of the 2001 Joint Statistical Meetings, 2001.
[25] C. E. Rasmussen and Z. Ghahramani. Infinite mixtures of Gaussian process experts. In Advances in Neural Information Processing Systems, volume 14, 2002.
[26] A. E. Gelfand, A. Kottas, and S. N. MacEachern. Bayesian nonparametric spatial modeling with Dirichlet process mixing. Journal of the American Statistical Association, 100(471):1021–1035, 2005. 9