iccv iccv2013 iccv2013-20 iccv2013-20-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zhaowen Wang, Jianchao Yang, Nasser Nasrabadi, Thomas Huang
Abstract: Sparse Representation-based Classification (SRC) is a powerful tool in distinguishing signal categories which lie on different subspaces. Despite its wide application to visual recognition tasks, current understanding of SRC is solely based on a reconstructive perspective, which neither offers any guarantee on its classification performance nor provides any insight on how to design a discriminative dictionary for SRC. In this paper, we present a novel perspective towards SRC and interpret it as a margin classifier. The decision boundary and margin of SRC are analyzed in local regions where the support of sparse code is stable. Based on the derived margin, we propose a hinge loss function as the gauge for the classification performance of SRC. A stochastic gradient descent algorithm is implemented to maximize the margin of SRC and obtain more discriminative dictionaries. Experiments validate the effectiveness of the proposed approach in predicting classification performance and improving dictionary quality over reconstructive ones. Classification results competitive with other state-ofthe-art sparse coding methods are reported on several data sets.
[1] M. Aharon, M. Elad, and A. Bruckstein. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Proc., 54(11):43 11–4322, 2006.
[2] B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In the 5th Annual Workshop on Computational Learning Theory, pages 144– 152, 1992.
[3] L. Bottou. Stochastic learning. In O. Bousquet and U. von Luxburg, editors, Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3 176, pages 146–168. Springer Verlag, 2004.
[4] D. M. Bradley and J. A. Bagnell. Differential sparse coding. In Adv. NIPS, pages 113–120, 2008.
[5] C.-K. Chiang, C.-H. Duan, S.-H. Lai, and S.-F. Chang. Learning component-level sparse representation using histogram information for image classification. In Proc. ICCV, pages 1519–1526, 2011.
[6] S. F. Cotter. Sparse representation for accurate classification of corrupted and occluded facial expressions. In Proc. ICASSP, pages 838–841, 2010.
[7] C. Domeniconi, J. Peng, and D. Gunopulos. Locally adaptive metric nearest-neighbor classification. IEEE Trans. PAMI, 24(9): 1281–1285, 2002.
[8] K. Engan, S. O. Aase, and J. Hakon Husoy. Method of optimal directions for frame design. In Proc. ICASSP, pages 2443–2446, 1999.
[9] A. Georghiades, P. Belhumeur, and D. Kriegman. From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. PAMI, 23(6):643–660, 2001.
[10] Z. Jiang, Z. Lin, and L. S. Davis. Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In Proc. CVPR, pages 1697–1704, 2011.
[11] T. Kohonen. Improved versions of learning vector quanti-
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22] zation. In IJCNN International Joint Conference on Neural Networks, volume 1, pages 545–550, 1990. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278–2324, 1998. H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In Adv. NIPS, pages 801–808, 2007. J. Mairal, F. Bach, and J. Ponce. Task-driven dictionary learning. IEEE Trans. PAMI, 32(4), 2012. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online dictionary learning for sparse coding. In Proc. ICML, pages 689–696, 2009. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. Discriminative learned dictionaries for local image analysis. In Proc. CVPR, pages 1–8, 2008. A. Martinez and R. Benavente. The AR face database. CVC Technical Report 24, 1998. N. Mehta and A. Gray. Sparsity-based generalization bounds for predictive sparse coding. In Proc. ICML, 2013. accepted. Q. Qiu, Z. Jiang, and R. Chellappa. Sparse dictionary-based representation and recognition of action attributes. In Proc. ICCV, pages 707–714, 2011. I. Ramirez, P. Sprechmann, and G. Sapiro. Classification and clustering via dictionary learning with structured incoherence and shared features. In Proc. CVPR, pages 3501– 3508, 2010. M. Soltanolkotabi and E. J. Candes. A geometric analysis of subspace clustering with outliers. arXiv preprint arXiv:1112.4258, 2011. V. Vapnik. The nature of statistical learning theory. springer, 1999.
[23] K. Q. Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In Adv. NIPS, pages 1473–1480, 2006.
[24] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma. Robust face recognition via sparse representation. IEEE Trans. PAMI, 31(2):210–227, 2009.
[25] J. Yang, J. Wang, and T. S. Huang. Learning the sparse representation for classification. In Proc. ICME, pages 1–6, 2011.
[26] J. Yang, Z. Wang, Z. Lin, X. Shu, and T. Huang. Bilevel sparse coding for coupled feature spaces. In Proc. CVPR, 2012.
[27] M. Yang, L. Zhang, X. Feng, and D. Zhang. Fisher discrimination dictionary learning for sparse representation. In Proc. ICCV, pages 543–550, 2011.
[28] L. Zhang, M. Yang, and X. Feng. Sparse representation or collaborative representation: Which helps face recognition? In Proc. ICCV, pages 471–478, 2011.
[29] H. Zou, T. Hastie, and R. Tibshirani. On the “degrees of freedom” of the LASSO. The Annals of Statistics, 35(5):2173– 2192, 2007. 1224