nips nips2008 nips2008-80 nips2008-80-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jihun Hamm, Daniel D. Lee
Abstract: Subspace-based learning problems involve data whose elements are linear subspaces of a vector space. To handle such data structures, Grassmann kernels have been proposed and used previously. In this paper, we analyze the relationship between Grassmann kernels and probabilistic similarity measures. Firstly, we show that the KL distance in the limit yields the Projection kernel on the Grassmann manifold, whereas the Bhattacharyya kernel becomes trivial in the limit and is suboptimal for subspace-based problems. Secondly, based on our analysis of the KL distance, we propose extensions of the Projection kernel which can be extended to the set of affine as well as scaled subspaces. We demonstrate the advantages of these extended kernels for classification and recognition tasks with Support Vector Machines and Kernel Discriminant Analysis using synthetic and real image databases. 1
[1] Gianfranco Doretto, Alessandro Chiuso, Ying Nian Wu, and Stefano Soatto. Dynamic textures. Int. J. Comput. Vision, 51(2):91–109, 2003.
[2] Alan Edelman, Tom´ s A. Arias, and Steven T. Smith. The geometry of algorithms with orthogonality a constraints. SIAM J. Matrix Anal. Appl., 20(2):303–353, 1999.
[3] Athinodoros S. Georghiades, Peter N. Belhumeur, and David J. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):643–660, 2001.
[4] Zoubin Ghahramani and Geoffrey E. Hinton. The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, Department of Computer Science, University of Toronto, 21 1996.
[5] Jihun Hamm. Subspace-based Learning with Grassmann Manifolds. Electrical and Systems Engineering, University of Pennsylvania, 2008. http://www.seas.upenn.edu/ jhham/Papers/thesis-jh.pdf. Ph.D thesis in Available at
[6] Jihun Hamm and Daniel Lee. Grassmann discriminant analysis: a unifying view on subspace-based learning. In Int. Conf. Mach. Learning, 2008. 7 Yale Face 100 rate (%) 90 Eucl Lin LinSc Aff AffSc BC Bhat 80 70 60 50 40 1 3 5 7 9 subspace dimension (m) ETH!80 100 rate (%) 90 Eucl Lin LinSc Aff AffSc BC Bhat 80 70 60 50 40 1 3 5 7 9 subspace dimension (m) Figure 1: Comparison of Grassmann kernels for face recognition/ object categorization tasks with kernel discriminant analysis. The extended Projection kernels (Lin/LinSc/Aff/ AffSc) outperform the baseline method (Eucl) and the Binet-Cauchy (BC) and the Bhattacharyya (Bhat) kernels.
[7] Tony Jebara and Risi Imre Kondor. Bhattacharyya expected likelihood kernels. In COLT, pages 57–71, 2003.
[8] Risi Imre Kondor and Tony Jebara. A kernel between sets of vectors. In Proc. of the 20th Int. Conf. on Mach. Learn., pages 361–368, 2003.
[9] Bastian Leibe and Bernt Schiele. Analyzing appearance and contour based methods for object categorization. CVPR, 02:409, 2003.
[10] Bernhard Sch¨ lkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularo ization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001.
[11] Gregory Shakhnarovich, John W. Fisher, and Trevor Darrell. Face recognition from long-term observations. In Proc. of the 7th Euro. Conf. on Computer Vision, pages 851–868, London, UK, 2002.
[12] Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. Journal Of The Royal Statistical Society Series B, 61(3):611–622, 1999.
[13] Pavan Turaga, Ashok Veeraraghavan, and Rama Chellappa. Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision. In CVPR, 2008.
[14] Ashok Veeraraghavan, Amit K. Roy-Chowdhury, and Rama Chellappa. Matching shape sequences in video with applications in human movement analysis. IEEE Trans. Pattern Anal. Mach. Intell., 27(12):1896–1909, 2005.
[15] S.V.N. Vishwanathan and Alexander J. Smola. Binet-Cauchy kernels. In NIPS, 2004.
[16] Liwei Wang, Xiao Wang, and Jufu Feng. Subspace distance analysis with application to adaptive bayesian algorithm for face recognition. Pattern Recogn., 39(3):456–464, 2006.
[17] Lior Wolf and Amnon Shashua. Learning over sets using kernel principal angles. J. Mach. Learn. Res., 4:913–931, 2003.
[18] Shaohua Kevin Zhou and Rama Chellappa. From sample similarity to ensemble similarity: Probabilistic distance measures in Reproducing Kernel Hilbert Space. IEEE Trans. Pattern Anal. Mach. Intell., 28(6):917–929, 2006. 8