nips nips2004 nips2004-42 nips2004-42-reference knowledge-graph by maker-knowledge-mining

42 nips-2004-Computing regularization paths for learning multiple kernels

Source: pdf

Author: Francis R. Bach, Romain Thibaux, Michael I. Jordan

Abstract: The problem of learning a sparse conic combination of kernel functions or kernel matrices for classification or regression can be achieved via the regularization by a block 1-norm [1]. In this paper, we present an algorithm that computes the entire regularization path for these problems. The path is obtained by using numerical continuation techniques, and involves a running time complexity that is a constant times the complexity of solving the problem for one value of the regularization parameter. Working in the setting of kernel linear regression and kernel logistic regression, we show empirically that the effect of the block 1-norm regularization differs notably from the (non-block) 1-norm regularization commonly used for variable selection, and that the regularization path is of particular value in the block case. 1

reference text

[1] F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality, and the SMO algorithm. In ICML, 2004.

[2] G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan. Learning the kernel matrix with semidefinite programming. JMLR, 5:27–72, 2004.

[3] C. S. Ong, A. J. Smola, and R. C. Williamson. Hyperkernels. In NIPS 15, 2003.

[4] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. Ann. Stat., 32(2):407–499, 2004.

[5] T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu. The entire regularization path for the support vector machine. In NIPS 17, 2005.

[6] A. Corduneanu and T. Jaakkola. Continuation methods for mixing heterogeneous sources. In UAI, 2002.

[7] T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer-Verlag, 2001.

[8] E. L. Allgower and K. Georg. Continuation and path following. Acta Numer., 2:1–64, 1993.

[9] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge Univ. Press, 2003.

[10] J. F. Bonnans, J. C. Gilbert, C. Lemar´ chal, and C. A. Sagastizbal. Numerical Optimization e Theoretical and Practical Aspects. Springer, 2003.