nips nips2013 nips2013-285 nips2013-285-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yuhong Guo
Abstract: Principal component analysis (PCA), a well-established technique for data analysis and processing, provides a convenient form of dimensionality reduction that is effective for cleaning small Gaussian noises presented in the data. However, the applicability of standard principal component analysis in real scenarios is limited by its sensitivity to large errors. In this paper, we tackle the challenge problem of recovering data corrupted with errors of high magnitude by developing a novel robust transfer principal component analysis method. Our method is based on the assumption that useful information for the recovery of a corrupted data matrix can be gained from an uncorrupted related data matrix. Specifically, we formulate the data recovery problem as a joint robust principal component analysis problem on the two data matrices, with common principal components shared across matrices and individual principal components specific to each data matrix. The formulated optimization problem is a minimization problem over a convex objective function but with non-convex rank constraints. We develop an efficient proximal projected gradient descent algorithm to solve the proposed optimization problem with convergence guarantees. Our empirical results over image denoising tasks show the proposed method can effectively recover images with random large errors, and significantly outperform both standard PCA and robust PCA with rank constraints. 1
[1] F. Bach. Consistency of trace norm minimization. Journal of Machine Learning Research, 9:1019–1048, 2008.
[2] A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences, 2, No. 1:183–202, 2009.
[3] M Fazel. Matrix Rank Minimization with Applications. PhD thesis, Stanford University, 2002.
[4] S. Gupta, D. Phung, B. Adams, and S. Venkatesh. Regularized nonnegative shared subspace learning. Data Mining and Knowledge Discovery, 26:57–97, 2013.
[5] P. Huber. Robust Statistics. New York, New York, 1981.
[6] S. Ji and J. Ye. An accelerated gradient method for trace norm minimization. In Proc. of International Conference on Machine Learning (ICML), 2009.
[7] I. Jolliffe. Principal Component Analysis. Springer-Verlag, New York, New York, 1986.
[8] Q. Ke and T. Kanade. Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005.
[9] S. Ma, D. Goldfarb, and L. Chen. Fixed point and bregman iterative methods for matrix rank minimization. Mathematical Programming, 2009.
[10] S. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010.
[11] B. Recht, M. Fazel, and P. Parrilo. Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Review, 52, no 3:471–501, 2010.
[12] M. Tipping and C. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, B, 6(3):611–622, 1999.
[13] F. De La Torre and M. Black. A framework for robust subspace learning. International Journal of Computer Vision (IJCV), 54(1-3):117–142, 2003.
[14] J. Wright, Y. Peng, Y. Ma, A. Ganesh, and S. Rao. Robust principal component analysis: Exact recovery of corrupted low-rank matrices by convex optimization. In Advances in Neural Information Processing Systems (NIPS), 2009.
[15] X. Zhang, Y. Yu, M. White, R. Huang, and D. Schuurmans. Convex sparse coding, subspace learning, and semi-supervised extensions. In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2011. 9