nips nips2012 nips2012-277 nips2012-277-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: S. D. Babacan, Shinichi Nakajima, Minh Do
Abstract: In this paper, we consider the problem of clustering data points into lowdimensional subspaces in the presence of outliers. We pose the problem using a density estimation formulation with an associated generative model. Based on this probability model, we first develop an iterative expectation-maximization (EM) algorithm and then derive its global solution. In addition, we develop two Bayesian methods based on variational Bayesian (VB) approximation, which are capable of automatic dimensionality selection. While the first method is based on an alternating optimization scheme for all unknowns, the second method makes use of recent results in VB matrix factorization leading to fast and effective estimation. Both methods are extended to handle sparse outliers for robustness and can handle missing values. Experimental results suggest that proposed methods are very effective in subspace clustering and identifying outliers. 1
[1] S. D. Babacan, M. Luessi, R. Molina, and A. K. Katsaggelos. Sparse Bayesian methods for low-rank matrix estimation. IEEE Trans. Signal Proc., 60(8), Aug 2012.
[2] C. M. Bishop. Bayesian principal components. In NIPS, volume 11, pages 382–388, 1999.
[3] C. M. Bishop. Variational principal components. In Proc. of ICANN, volume 1, pages 514–509, 1999.
[4] C.M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
[5] E. J. Cand` s, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? CoRR, abs/0912.3599, e 2009.
[6] J. P. Costeira and T. Kanade. A multibody factorization method for independently moving objects. Int. J. Comput. Vision, 29(3):159–179, September 1998.
[7] E. Elhamifar and R. Vidal. Sparse subspace clustering. In CVPR, pages 2790–2797, 2009.
[8] P. Favaro, R. Vidal, and A. Ravichandran. A closed form solution to robust subspace estimation and clustering. In CVPR, pages 1801–1807, 2011.
[9] Z. Ghahramani and M. J. Beal. Variational inference for Bayesian mixtures of factor analysers. In NIPS, volume 12, pages 449–455, 2000.
[10] A. K. Gupta and D. K. Nagar. Matrix Variate Distributions. Chapman & Hall/CRC, New York, 2000.
[11] K. Huang and S. Aviyente. Sparse representation for signal classification. In NIPS, 2006.
[12] K.-C. Lee, J. Ho, and D. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Machine Intell., 27:684–698, 2005.
[13] Y. J. Lim and T. W. Teh. Variational Bayesian approach to movie rating prediction. In Proc. of KDD Cup and Workshop, 2007.
[14] G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma. Robust recovery of subspace structures by low-rank representation. CoRR, abs/1010.2955, 2012.
[15] G. Liu, Z. Lin, and Y. Yu. Robust subspace segmentation by low-rank representation. In ICML, pages 663–670, 2010.
[16] G. Liu, H. Xu, and S. Yan. Exact subspace segmentation and outlier detection by low-rank representation. In AISTATS, 2012.
[17] G. Liu and S. Yan. Latent low-rank representation for subspace segmentation and feature extraction. In ICCV, 2011.
[18] U. Luxburg. A tutorial on spectral clustering. Statistics and Computing, 17(4):395–416, December 2007.
[19] Y. Ma, A. Yang, H. Derksen, and R. Fossum. Estimation of subspace arrangements with applications in modeling and segmenting mixed data,. SIAM Review, 50(3):413–458, 2008.
[20] S. Nakajima and M. Sugiyama. Theoretical analysis of Bayesian matrix factorization. Journal of Machine Learning Research, 12:2583–2648, 2011.
[21] R. M. Neal. Bayesian Learning for Neural Networks. Springer, 1996.
[22] H. Peterkriegel, P. Kroger, and A. Zimek. Clustering high-dimensional data: a survey on subspace slustering, pattern-based clustering, and correlation clustering. In Proc. KDD, 2008.
[23] S. Rao, R. Tron, R. Vidal, and Y. Ma. Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Trans. Pattern Anal. Machine Intell., 32(10):1832–1845, 2010.
[24] J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Machine Intell., 22(8):888 –905, aug 2000.
[25] M. Soltanolkotabi and E. J. Cand` s. A geometric analysis of subspace clustering with outliers. CoRR, e 2011.
[26] M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analyzers. Neural Comput., 11(2):443–482, February 1999.
[27] R. Tron and R. Vidal. A benchmark for the comparison of 3-d motion segmentation algorithms. In CVPR, June 2007.
[28] R. Vidal. Subspace clustering. IEEE Signal Process. Mag., 28(2):52–68, 2011.
[29] R. Vidal, Y. Ma, and S. Sastry. Generalized principal component analysis (gpca). IEEE Trans. on PAMI, 27(12):1945–1959, 2005.
[30] J. Yan and M. Pollefeys. A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In ECCV, volume 4, pages 94–106, 2006.
[31] C. Zhang and R. R. Bitmead. Subspace system identification for training-based MIMO channel estimation. Automatica, 41:1623–1632, 2005. 9