nips nips2007 nips2007-118 nips2007-118-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Christian Walder, Olivier Chapelle
Abstract: This paper considers kernels invariant to translation, rotation and dilation. We show that no non-trivial positive definite (p.d.) kernels exist which are radial and dilation invariant, only conditionally positive definite (c.p.d.) ones. Accordingly, we discuss the c.p.d. case and provide some novel analysis, including an elementary derivation of a c.p.d. representer theorem. On the practical side, we give a support vector machine (s.v.m.) algorithm for arbitrary c.p.d. kernels. For the thinplate kernel this leads to a classifier with only one parameter (the amount of regularisation), which we demonstrate to be as effective as an s.v.m. with the Gaussian kernel, even though the Gaussian involves a second parameter (the length scale). 1
Boughorbel, S., Tarel, J.-P., & Boujemaa, N. (2005). Conditionally positive definite kernels for svm based image recognition. Proc. of IEEE ICME’05. Amsterdam. Chapelle, O. (2007). Training a support vector machine in the primal. Neural Computation, 19, 1155–1178. Chapelle, O., & Sch¨ lkopf, B. (2001). Incorporating invariances in nonlinear support vector machines. In o T. Dietterich, S. Becker and Z. Ghahramani (Eds.), Advances in neural information processing systems 14, 609–616. Cambridge, MA: MIT Press. Fleuret, F., & Sahbi, H. (2003). Scale-invariance of support vector machines based on the triangular kernel. Proc. of ICCV SCTV Workshop. Golub, G. H., & Van Loan, C. F. (1996). Matrix computations. Baltimore MD: The Johns Hopkins University Press. 2nd edition. Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support vector classification (Technical Report). National Taiwan University. Mika, S., R¨ tsch, G., Weston, J., Sch¨ lkopf, B., Smola, A., & M¨ ller, K.-R. (2003). Constructing descriptive a o u and discriminative non-linear features: Rayleigh coefficients in feature spaces. IEEE PAMI, 25, 623–628. Sch¨ lkopf, B., & Smola, A. J. (2002). Learning with kernels: Support vector machines, regularization, optio mization, and beyond. Cambridge: MIT Press. Smola, A., Sch¨ lkopf, B., & M¨ ller, K.-R. (1998). The connection between regularization operators and support o u vector kernels. Neural Networks, 11, 637–649. Wahba, G. (1990). Spline models for observational data. Philadelphia: Series in Applied Math., Vol. 59, SIAM. Walder, C., & Chapelle, O. (2007). Learning with transformation invariant kernels (Technical Report 165). Max Planck Institute for Biological Cybernetics, Department of Empirical Inference, T¨ bingen, Germany. u Wendland, H. (2004). Scattered data approximation. Monographs on Applied and Computational Mathematics. Cambridge University Press. 8