nips nips2009 nips2009-195 nips2009-195-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Wu-jun Li, Dit-Yan Yeung, Zhihua Zhang
Abstract: One crucial assumption made by both principal component analysis (PCA) and probabilistic PCA (PPCA) is that the instances are independent and identically distributed (i.i.d.). However, this common i.i.d. assumption is unreasonable for relational data. In this paper, by explicitly modeling covariance between instances as derived from the relational information, we propose a novel probabilistic dimensionality reduction method, called probabilistic relational PCA (PRPCA), for relational data analysis. Although the i.i.d. assumption is no longer adopted in PRPCA, the learning algorithms for PRPCA can still be devised easily like those for PPCA which makes explicit use of the i.i.d. assumption. Experiments on realworld data sets show that PRPCA can effectively utilize the relational information to dramatically outperform PCA and achieve state-of-the-art performance. 1
[1] D. J. Bartholomew and M. Knott. Latent Variable Models and Factor Analysis. Kendall’s Library of Statistics,7, second edition, 1999.
[2] C. M. Bishop. Bayesian PCA. In NIPS 11, 1998.
[3] J. Chang and D. M. Blei. Relational topic models for document networks. In AISTATS, 2009.
[4] J. Chang, J. L. Boyd-Graber, and D. M. Blei. Connections between the lines: augmenting social networks with text. In KDD, pages 169–178, 2009.
[5] W. Chu, V. Sindhwani, Z. Ghahramani, and S. S. Keerthi. Relational learning with Gaussian processes. In NIPS 19, 2007.
[6] F. Chung. Spectral Graph Theory. Number 92 in Regional Conference Series in Mathematics. American Mathematical Society, 1997.
[7] D. A. Cohn and T. Hofmann. The missing link - a probabilistic model of document content and hypertext connectivity. In NIPS 13, 2000.
[8] M. Craven, D. DiPasquo, D. Freitag, A. McCallum, T. M. Mitchell, K. Nigam, and S. Slattery. Learning to extract symbolic knowledge from the world wide web. In AAAI/IAAI, pages 509– 516, 1998.
[9] A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1–38, 1977.
[10] L. Getoor and B. Taskar. Introduction to Statistical Relational Learning. The MIT Press, 2007.
[11] A. K. Gupta and D. K. Nagar. Matrix Variate Distributions. Chapman & Hall/CRC, 2000.
[12] H. Howard. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 27:417–441, 1933.
[13] I. T. Jolliffe. Principal Component Analysis. Springer, second edition, 2002.
[14] W.-J. Li and D.-Y. Yeung. Relation regularized matrix factorization. In IJCAI, 2009.
[15] W.-J. Li, Z. Zhang, and D.-Y. Yeung. Latent Wishart processes for relational kernel learning. In AISTATS, pages 336–343, 2009.
[16] A. McCallum, K. Nigam, J. Rennie, and K. Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127–163, 2000.
[17] R. Nallapati, A. Ahmed, E. P. Xing, and W. W. Cohen. Joint latent topic models for text and citations. In KDD, pages 542–550, 2008.
[18] C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
[19] R. Silva, W. Chu, and Z. Ghahramani. Hidden common cause relations in relational learning. In NIPS 20. 2008.
[20] B. Taskar, P. Abbeel, and D. Koller. Discriminative probabilistic models for relational data. In UAI, pages 485–492, 2002.
[21] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal Of The Royal Statistical Society Series B, 61(3):611–622, 1999.
[22] J.-P. Vert. Reconstruction of biological networks by supervised machine learning approaches. In Elements of Computational Systems Biology, 2009.
[23] T. Yang, R. Jin, Y. Chi, and S. Zhu. A Bayesian framework for community detection integrating content and link. In UAI, 2009.
[24] T. Yang, R. Jin, Y. Chi, and S. Zhu. Combining link and content for community detection: a discriminative approach. In KDD, pages 927–936, 2009.
[25] D. Zhou, B. Sch¨ lkopf, and T. Hofmann. Semi-supervised learning on directed graphs. In o NIPS 17, 2004.
[26] S. Zhu, K. Yu, Y. Chi, and Y. Gong. Combining content and link for classification using matrix factorization. In SIGIR, 2007. 9