nips nips2009 nips2009-155 nips2009-155-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ilya Sutskever, Joshua B. Tenenbaum, Ruslan Salakhutdinov
Abstract: We consider the problem of learning probabilistic models for complex relational structures between various types of objects. A model can help us “understand” a dataset of relational facts in at least two ways, by finding interpretable structure in the data, and by supporting predictions, or inferences about whether particular unobserved relations are likely to be true. Often there is a tradeoff between these two aims: cluster-based models yield more easily interpretable representations, while factorization-based approaches have given better predictive performance on large data sets. We introduce the Bayesian Clustered Tensor Factorization (BCTF) model, which embeds a factorized representation of relations in a nonparametric Bayesian clustering framework. Inference is fully Bayesian but scales well to large data sets. The model simultaneously discovers interpretable clusters and yields predictive performance that matches or beats previous probabilistic models for relational data.
[1] Edoardo Airoldi, David M. Blei, Stephen E. Fienberg, and Eric P. Xing. Mixed membership stochastic blockmodels. In NIPS, pages 33–40. MIT Press, 2008.
[2] P.J. Carrington, J. Scott, and S. Wasserman. Models and methods in social network analysis. Cambridge University Press, 2005.
[3] W. Chu and Z. Ghahramani. Probabilistic models for incomplete multi-dimensional arrays. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 5, 2009.
[4] W. Denham. The Detection of Patterns in Alyawarra Nonverbal Behavior. PhD thesis, Department of Anthropology, University of Washington, 1973.
[5] S. Jain and R.M. Neal. A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model. Journal of Computational and Graphical Statistics, 13(1):158–182, 2004.
[6] Y. Katz, N.D. Goodman, K. Kersting, C. Kemp, and J.B. Tenenbaum. Modeling Semantic Cognition as Logical Dimensionality Reduction. In Proceedings of Thirtieth Annual Meeting of the Cognitive Science Society, 2008.
[7] C. Kemp, N.D. Goodman, and J.B. Tenenbaum. Theory acquisition and the language of thought. In Proceedings of Thirtieth Annual Meeting of the Cognitive Science Society, 2008.
[8] C. Kemp, J.B. Tenenbaum, T.L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an infinite relational model. In Proceedings of the National Conference on Artificial Intelligence, volume 21, page 381. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.
[9] S. Kok and P. Domingos. Statistical predicate invention. In Proceedings of the 24th international conference on Machine learning, pages 433–440. ACM New York, NY, USA, 2007.
[10] H. Liu and P. Singh. ConceptNeta practical commonsense reasoning tool-kit. BT Technology Journal, 22(4):211–226, 2004.
[11] A.T. McCray. An upper-level ontology for the biomedical domain. Comparative and Functional Genomics, 4(1):80–84, 2003.
[12] R.M. Neal. Probabilistic inference using Markov chain Monte Carlo methods, 1993.
[13] R.M. Neal. Markov chain sampling methods for Dirichlet process mixture models. Journal of computational and graphical statistics, pages 249–265, 2000.
[14] Ian Porteous, Evgeniy Bart, and Max Welling. Multi-HDP: A non parametric bayesian model for tensor factorization. In Dieter Fox and Carla P. Gomes, editors, AAAI, pages 1487–1490. AAAI Press, 2008.
[15] J. Riedl, J. Konstan, S. Lam, and J. Herlocker. Movielens collaborative filtering data set, 2006.
[16] J.F. Rual, K. Venkatesan, T. Hao, T. Hirozane-Kishikawa, A. Dricot, N. Li, G.F. Berriz, F.D. Gibbons, M. Dreze, N. Ayivi-Guedehoussou, et al. Towards a proteome-scale map of the human protein–protein interaction network. Nature, 437(7062):1173–1178, 2005.
[17] R. Salakhutdinov and A. Mnih. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proceedings of the 25th international conference on Machine learning, pages 880–887. ACM New York, NY, USA, 2008.
[18] R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. Advances in neural information processing systems, 20, 2008.
[19] R. Speer, C. Havasi, and H. Lieberman. AnalogySpace: Reducing the dimensionality of common sense knowledge. In Proceedings of AAAI, 2008. 8