nips nips2009 nips2009-114 nips2009-114-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yee W. Teh, Dilan Gorur
Abstract: The Indian buffet process (IBP) is an exchangeable distribution over binary matrices used in Bayesian nonparametric featural models. In this paper we propose a three-parameter generalization of the IBP exhibiting power-law behavior. We achieve this by generalizing the beta process (the de Finetti measure of the IBP) to the stable-beta process and deriving the IBP corresponding to it. We find interesting relationships between the stable-beta process and the Pitman-Yor process (another stochastic process used in Bayesian nonparametric models with interesting power-law properties). We derive a stick-breaking construction for the stable-beta process, and find that our power-law IBP is a good model for word occurrences in document corpora. 1
[1] T. L. Griffiths and Z. Ghahramani. Infinite latent feature models and the Indian buffet process. In Advances in Neural Information Processing Systems, volume 18, 2006.
[2] Z. Ghahramani, T. L. Griffiths, and P. Sollich. Bayesian nonparametric latent feature models (with discussion and rejoinder). In Bayesian Statistics, volume 8, 2007.
[3] D. Knowles and Z. Ghahramani. Infinite sparse factor analysis and infinite independent components analysis. In International Conference on Independent Component Analysis and Signal Separation, volume 7 of Lecture Notes in Computer Science. Springer, 2007.
[4] D. G¨ r¨ r, F. J¨ kel, and C. E. Rasmussen. A choice model with infinitely many latent features. ou a In Proceedings of the International Conference on Machine Learning, volume 23, 2006.
[5] D. J. Navarro and T. L. Griffiths. Latent features in similarity judgment: A nonparametric Bayesian approach. Neural Computation, in press 2008.
[6] E. Meeds, Z. Ghahramani, R. M. Neal, and S. T. Roweis. Modeling dyadic data with binary latent factors. In Advances in Neural Information Processing Systems, volume 19, 2007.
[7] F. Wood, T. L. Griffiths, and Z. Ghahramani. A non-parametric Bayesian method for inferring hidden causes. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, volume 22, 2006.
[8] S. Goldwater, T.L. Griffiths, and M. Johnson. Interpolating between types and tokens by estimating power-law generators. In Advances in Neural Information Processing Systems, volume 18, 2006.
[9] Y. W. Teh. A hierarchical Bayesian language model based on Pitman-Yor processes. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 985–992, 2006.
[10] E. Sudderth and M. I. Jordan. Shared segmentation of natural scenes using dependent PitmanYor processes. In Advances in Neural Information Processing Systems, volume 21, 2009.
[11] M. Perman, J. Pitman, and M. Yor. Size-biased sampling of Poisson point processes and excursions. Probability Theory and Related Fields, 92(1):21–39, 1992.
[12] J. Pitman and M. Yor. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25:855–900, 1997.
[13] H. Ishwaran and L. F. James. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96(453):161–173, 2001.
[14] T. S. Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2):209–230, 1973.
[15] N. L. Hjort. Nonparametric Bayes estimators based on beta processes in models for life history data. Annals of Statistics, 18(3):1259–1294, 1990.
[16] R. Thibaux and M. I. Jordan. Hierarchical beta processes and the Indian buffet process. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, volume 11, pages 564–571, 2007.
[17] M. Perman. Random Discrete Distributions Derived from Subordinators. PhD thesis, Department of Statistics, University of California at Berkeley, 1990.
[18] J. F. C. Kingman. Completely random measures. Pacific Journal of Mathematics, 21(1):59–78, 1967.
[19] J. F. C. Kingman. Poisson Processes. Oxford University Press, 1993.
[20] Y. Kim. Nonparametric Bayesian estimators for counting processes. Annals of Statistics, 27(2):562–588, 1999.
[21] Y. W. Teh, D. G¨ r¨ r, and Z. Ghahramani. Stick-breaking construction for the Indian buffet proou cess. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 11, 2007.
[22] R. L. Wolpert and K. Ickstadt. Simulations of l´ vy random fields. In Practical Nonparametric e and Semiparametric Bayesian Statistics, pages 227–242. Springer-Verlag, 1998.
[23] G. Zipf. Selective Studies and the Principle of Relative Frequency in Language. Harvard University Press, Cambridge, MA, 1932. 9