nips nips2013 nips2013-244 nips2013-244-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ichiro Takeuchi, Tatsuya Hongo, Masashi Sugiyama, Shinichi Nakajima
Abstract: We introduce an extended formulation of multi-task learning (MTL) called parametric task learning (PTL) that can systematically handle infinitely many tasks parameterized by a continuous parameter. Our key finding is that, for a certain class of PTL problems, the path of the optimal task-wise solutions can be represented as piecewise-linear functions of the continuous task parameter. Based on this fact, we employ a parametric programming technique to obtain the common shared representation across all the continuously parameterized tasks. We show that our PTL formulation is useful in various scenarios such as learning under non-stationarity, cost-sensitive learning, and quantile regression. We demonstrate the advantage of our approach in these scenarios.
[1] A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In Advances in Neural Information Processing Systems, volume 19, pages 41–48. 2007.
[2] A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying. A spectral regularization framework for multi-task structure learning. In Advances in Neural Information Processing Systems, volume 20, pages 25–32. 2008.
[3] L. Cao and F. Tay. Support vector machine with adaptive parameters in finantial time series forecasting. IEEE Transactions on Neural Networks, 14(6):1506–1518, 2003.
[4] F. R. Bach, D. Heckerman, and E. Horvits. Considering cost asymmetry in learning classifiers. Journal of Machine Learning Research, 7:1713–41, 2006.
[5] R. Koenker. Quantile Regression. Cambridge University Press, 2005.
[6] K. Ritter. On parametric linear and quadratic programming problems. mathematical Programming: Proceedings of the International Congress on Mathematical Programming, pages 307–335, 1984.
[7] E. L. Allgower and K. George. Continuation and path following. Acta Numerica, 2:1–63, 1993.
[8] T. Gal. Postoptimal Analysis, Parametric Programming, and Related Topics. Walter de Gruyter, 1995.
[9] M. J. Best. An algorithm for the solution of the parametric quadratic programming problem. Applied Mathemetics and Parallel Computing, pages 57–76, 1996.
[10] M. Fazel, H. Hindi, and S. P. Boyd. A rank minimization heuristic with application to minimum order system approximation. In Proceedings of the American Control Conference, volume 6, pages 4734–4739, 2001.
[11] B. A. Turlach, W. N. Venables, and S. J. Wright. Simultaneous variable selection. Technometrics, 47:349–363, 2005.
[12] G. Obozinski, B. Taskar, and M. Jordan. Joint covariate selection and joint sbspace selection for multiple classification problems. Statistics and Computing, 20(2):231–252, 2010.
[13] M. R. Osborne, B. Presnell, and B. A. Turlach. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 20(20):389–404, 2000.
[14] B. Efron and R. Tibshirani. Least angle regression. Annals of Statistics, 32(2):407–499, 2004.
[15] T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu. The entire regularization path for the support vector machine. Journal of Machine Learning Research, 5:1391–415, 2004.
[16] Y. Lin, Y. Lee, and G. Wahba. Support vector machines for classification in nonstandard situations. Machine Learning, 46:191–202, 2002.
[17] M. A. Davenport, R. G. Baraniuk, and C. D. Scott. Tuning support vector machine for minimax and Neyman-Pearson classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010.
[18] G. Lee and C. Scott. Nested support vector machines. IEEE Transactions on Signal Processing, 58(3):1648–1660, 2010.
[19] R. Koenker. Quantile Regression. Cambridge University Press, 2005.
[20] I. Takeuchi, Q. V. Le, T. Sears, and A. J. Smola. Nonparametric quantile estimation. Journal of Machine Learning Research, 7:1231–1264, 2006.
[21] L. K. Bachrach, T. Hastie, M. C. Wang, B. Narasimhan, and R. Marcus. Acquisition in healthy Asian, hispanic, black and caucasian youth. a longitudinal study. The Journal of Clinical Endocrinology and Metabolism, 84:4702–4712, 1999.
[22] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004. 9