nips nips2008 nips2008-31 nips2008-31-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Shakir Mohamed, Zoubin Ghahramani, Katherine A. Heller
Abstract: Principal Components Analysis (PCA) has become established as one of the key tools for dimensionality reduction when dealing with real valued data. Approaches such as exponential family PCA and non-negative matrix factorisation have successfully extended PCA to non-Gaussian data types, but these techniques fail to take advantage of Bayesian inference and can suffer from problems of overfitting and poor generalisation. This paper presents a fully probabilistic approach to PCA, which is generalised to the exponential family, based on Hybrid Monte Carlo sampling. We describe the model which is based on a factorisation of the observed data matrix, and show performance of the model on both synthetic and real data. 1
[1] C. M. Bishop, Pattern Recognition and Machine Learning. Information Science and Statistics, Springer, August 2006.
[2] D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” in Advances in Neural Information Processing Systems, vol. 13, pp. 556 – 562, MIT Press, Cambridge, MA, 2001.
[3] W. Buntine and A. Jakulin, “Discrete components analysis,” in Subspace, Latent Structure and Feature Selection, vol. 3940/2006, pp. 1–33, Springer (LNCS), 2006.
[4] M. Collins, S. Dasgupta, and R. Schapire, “A generalization of principal components to the exponential family,” in Advances in Neural Information Processing Systems, vol. 14, pp. 617 – 624, MIT Press, Cambridge, MA, 2002.
[5] Sajama and A. Orlitsky, “Semi-parametric exponential family PCA,” in Advances in Neural Information Processing Systems, vol. 17, pp. 1177 – 1184, MIT Press, Cambridge, MA, 2004.
[6] M. Welling, C. Chemudugunta, and N. Sutter, “Deterministic latent variable models and their pitfalls,” in SIAM Conference on Data Mining (SDM), pp. 196 – 207, 2008.
[7] R. M. Neal, “Probabilistic inference using Markov Chain Monte Carlo methods,” Tech. Rep. CRG-TR-93-1, University of Toronto, Department of Computer Science, 1993.
[8] C. Andrieu, N. De Freitas, A. Doucet, and M. I. Jordan, “An introduction to MCMC for machine learning,” Machine Learning, vol. 50, pp. 5–43, 2003.
[9] D. J. C. MacKay, Information Theory, Inference & Learning Algorithms. Cambridge University Press, June 2002.
[10] M. E. Tipping, “Probabilistic visualisation of high dimensional binary data,” in Advances in Neural Information Processing Systems, vol. 11, pp. 592 – 598, MIT Press, Cambridge, MA, 1999.
[11] “UCI machine learning repository.” http://archive.ics.uci.edu/ml/datasets/.