nips nips2004 nips2004-70 nips2004-70-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Saharon Rosset
Abstract: Regularization plays a central role in the analysis of modern data, where non-regularized fitting is likely to lead to over-fitted models, useless for both prediction and interpretation. We consider the design of incremental algorithms which follow paths of regularized solutions, as the regularization varies. These approaches often result in methods which are both efficient and highly flexible. We suggest a general path-following algorithm based on second-order approximations, prove that under mild conditions it remains “very close” to the path of optimal solutions and illustrate it with examples.
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13] Allgower, E. L. & Georg, K. (1993). Continuation and path following. Acta Numer., 2:164 Donoho, D., Johnstone, I., Kerkyachairan, G. & Picard, D. (1995). Wavelet shrinkage: Asymptopia? Annals of Statistics Efron, B., Hastie, T., Johnstone, I. & Tibshirani, R.(2004). Least Angle Regression. Annals of Statistics . Hastie, T., Rosset, S., Tibshirani, R. & Zhu, J. (2004). The Entire Regularization Path for the Support Vector Machine. Journal of Machine Learning Research, 5(Oct):1391–1415. Hastie, T., Tibshirani, R. & Friedman, J. (2001). Elements of Stat. Learning. Springer-Verlag Kim, Y & Kim, J. (2004) Gradient LASSO for feature selection. ICML-04, to appear. Rosset, S. (2003). Topics in Regularization and Boosting. PhD thesis, dept. of Statistics, Stanford University. http://www-stat.stanford.edu/˜saharon/papers/PhDThesis.pdf Rosset, S., Zhu, J. & Hastie,T. (2003). Boosting as a regularized path to a maximum margin classifier. Journal of Machine Learning Research, 5(Aug):941-973. Rosset, S. & Zhu, J. (2003). Piecewise linear regularized solution paths. Submitted. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. JRSSB Wahba, G., Gu, C., Wang, Y. & Chappell, R. (1995) Soft Classification, a.k.a. Risk Estimation, via Penalized Log Likelihood and Smoothing Spline Analysis of Variance. In D.H. Wolpert, editor, The Mathematics of Generalization. Zhu, J. & Hastie, T. (2003). Classification of Gene Microarrays by Penalized Logistic Regression. Biostatistics, to appear. Zhu, J., Hastie, T., Rosset, S. & Tibshirani, R. (2004). 1-norm support vector machines. Neural Information Processing Systems, 16.