nips nips2008 nips2008-200 nips2008-200-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Minh H. Nguyen, Fernando Torre
Abstract: Kernel Principal Component Analysis (KPCA) is a popular generalization of linear PCA that allows non-linear feature extraction. In KPCA, data in the input space is mapped to higher (usually) dimensional feature space where the data can be linearly modeled. The feature space is typically induced implicitly by a kernel function, and linear PCA in the feature space is performed via the kernel trick. However, due to the implicitness of the feature space, some extensions of PCA such as robust PCA cannot be directly generalized to KPCA. This paper presents a technique to overcome this problem, and extends it to a unified framework for treating noise, missing data, and outliers in KPCA. Our method is based on a novel cost function to perform inference in KPCA. Extensive experiments, in both synthetic and real data, show that our algorithm outperforms existing methods. 1
[1] Alzate, C. & Syukens, J.A. (2005) ‘Robust Kernel Principal Component Analysis uisng Huber’s Loss Function.’ 24th Benelux Meeting on Systems and Control.
[2] Bakir, G.H., Weston, J. & Sch¨ lkopf, B. (2004) ‘Learning to Find Pre-Images.’ in Thrun, S., Saul, L. & o Sch¨ lkopf, B. (Eds) Advances in Neural Information Processing Systems. o
[3] Berar, M., Desvignes, M., Bailly, G., Payan, Y. & Romaniuk, B. (2005) ‘Missing Data Estimation using Polynomial Kernels.’ Proceedings of International Conference on Advances in Pattern Recognition.
[4] Bishop, C.M., Svens´ n, M. & Williams, C.K.I. (1998) ‘GTM: The Generative Topographic Mapping.’ e Neural Computation, 10(1), 215–234.
[5] Black, M.J. & Anandan, P. (1996) ‘The Robust Estimation of Multiple Motions: Parametric and Piecewise-smooth Flow Fields.’ Computer Vision and Image Understanding, 63(1), 75–104.
[6] de la Torre, F. & Black, M.J. (2003) ‘A Framework for Robust Subspace Learning.’ International Journal of Computer Vision, 54(1–3), 117–142.
[7] Deng, X., Yuan, M. & Sudijanto, A. (2007) ‘A Note on Robust Principal Component Analysis.’ Contemporary Mathematics, 443, 21–33.
[8] Gross, R., Matthews, I., Cohn, J., Kanade, T. & Baker, S. (2007) ‘The CMU Multi-pose, Illumination, and Expression (Multi-PIE) Face Database.’ Technical report, Carnegie Mellon University.TR-07-08.
[9] Jolliffe, I. (2002) Principal Component Analysis. 2 edn. Springer-Verlag, New York.
[10] Kwok, J.T.Y. & Tsang, I.W.H. (2004) ‘The Pre-Image Problem in Kernel Methods.’ IEEE Transactions on Neural Networks, 15(6), 1517–1525.
[11] Lawrence, N.D. (2004) ‘Gaussian Process Latent Variable Models for Visualization of High Dimensional Data.’ in Thrun, S., Saul, L. & Sch¨ lkopf, B. (Eds) Advances in Neural Information Processing Systems. o
[12] Lu, C., Zhang, T., Zhang, R. & Zhang, C. (2003) ‘Adaptive Robust Kernel PCA Algorithm.’ Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing.
[13] Mika, S., Sch¨ lkopf, B., Smola, A., M¨ ller, K.R., Scholz, M. & R¨ tsch, G. (1999) ‘Kernel PCA and o u a De-Noising in Feature Spaces.’ Advances in Neural Information Processing Systems.
[14] Romdhani, S., Gong, S. & Psarrou, A. (1999) ‘Multi-view Nonlinear Active Shape Model Using Kernel PCA.’ British Machine Vision Conference, 483–492.
[15] Roweis, S. (1998) ‘EM Algorithms for PCA and SPCA.’ in Jordan, M., Kearns, M. & Solla, S. (Eds) Advances in Neural Information Processing Systems 10.
[16] Sanguinetti, G. & Lawrence, N.D. (2006) ‘Missing Data in Kernel PCA.’ Proceedings of European Conference on Machine Learning.
[17] Sch¨ lkopf, B., Mika, S., Smola, A., R¨ tsch, G. & M¨ ller, K.R. (1998) ‘Kernel PCA Pattern Reconstruco a u tion via Approximate Pre-Images.’ International Conference on Artificial Neural Networks.
[18] Sch¨ lkopf, B. & Smola, A. (2002) Learning with Kernels: Support Vector Machines, Regularization, o Optimization, and beyond. MIT Press, Cambridge, MA.
[19] Sch¨ lkopf, B., Smola, A. & Mller, K. (1998) ‘Nonlinear Component Analysis as a Kernel Eigenvalue o Problem.’ Neural Computation, 10, 1299–1319.
[20] Shawe-Taylor, J. & Cristianini, N. (2004) Kernel Methods for Pattern Analysis. Cambridge Uni. Press.
[21] Tipping, M. & Bishop, C.M. (1999) ‘Probabilistic Principal Component Analysis.’ Journal of the Royal Statistical Society B, 61, 611–622.
[22] Wang, L., Pang, Y.W., Shen, D.Y. & Yu, N.H. (2007) ‘An Iterative Algorithm for Robust Kernel Principal Component Analysis.’ Conference on Machine Learning and Cybernetics. 8