nips nips2006 nips2006-30 nips2006-30-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ingo Steinwart, Don Hush, Clint Scovel
Abstract: We establish a general oracle inequality for clipped approximate minimizers of regularized empirical risks and apply this inequality to support vector machine (SVM) type algorithms. We then show that for SVMs using Gaussian RBF kernels for classification this oracle inequality leads to learning rates that are faster than the ones established in [9]. Finally, we use our oracle inequality to show that a simple parameter selection approach based on a validation set can yield the same fast learning rates without knowing the noise exponents which were required to be known a-priori in [9]. 1
[1] P.L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inform. Theory, 44:525–536, 1998.
[2] G. Blanchard, O. Bousquet, and P. Massart. Statistical performance of support vector machines. Technical Report, 2004.
[3] O. Bousquet. A Bennet concentration inequality and its application to suprema of empirical processes. C. R. Math. Acad. Sci. Paris, 334:495–500, 2002.
[4] D.R. Chen, Q. Wu, Y.M. Ying, and D.X. Zhou. Support vector machine soft margin classifiers: Error analysis. Journal of Machine Learning Research, 5:1143–1175, 2004.
[5] N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, 2000.
[6] L. Devroye, L. Gy¨ rfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, New o York, 1996.
[7] F. Girosi, M. Jones, and T. Poggio. Regularization theory and neural networks architectures. Neural Computation, 7:219–269, 1995.
[8] S. Mendelson. Improving the sample complexity using global data. IEEE Trans. Inform. Theory, 48:1977–1991, 2002.
[9] I. Steinwart and C. Scovel. Fast rates for support vector machines using Gaussian kernels. Annals of Statistics, to appear.
[10] I. Steinwart and C. Scovel. Fast rates for support vector machines. In Proceedings of the 18th Annual Conference on Learning Theory, COLT 2005, pages 279–294. Springer, 2005.
[11] Q. Wu, Y. Ying, and D.-X. Zhou. Multi-kernel regularized classifiers. J. Complexity, to appear.