nips nips2013 nips2013-135 nips2013-135-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yu Zhang
Abstract: All the existing multi-task local learning methods are defined on homogeneous neighborhood which consists of all data points from only one task. In this paper, different from existing methods, we propose local learning methods for multitask classification and regression problems based on heterogeneous neighborhood which is defined on data points from all tasks. Specifically, we extend the knearest-neighbor classifier by formulating the decision function for each data point as a weighted voting among the neighbors from all tasks where the weights are task-specific. By defining a regularizer to enforce the task-specific weight matrix to approach a symmetric one, a regularized objective function is proposed and an efficient coordinate descent method is developed to solve it. For regression problems, we extend the kernel regression to multi-task setting in a similar way to the classification case. Experiments on some toy data and real-world datasets demonstrate the effectiveness of our proposed methods. 1
[1] R. K. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817–1853, 2005.
[2] A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In B. Sch¨ lkopf, J. C. Platt, and o T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 41–48, Vancouver, British Columbia, Canada, 2006.
[3] B. Bakker and T. Heskes. Task clustering and gating for bayesian multitask learning. Journal of Machine Learning Research, 4:83–99, 2003.
[4] J. Baxter. A Bayesian/information theoretic model of learning to learn via multiple task sampling. Machine Learning, 28(1):7–39, 1997.
[5] J. C. Bezdek and R. J. Hathaway. Convergence of alternating optimization. Neural, Parallel & Scientific Computations, 11(4):351–368, 2003.
[6] E. Bonilla, K. M. A. Chai, and C. Williams. Multi-task Gaussian process prediction. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 153–160, Vancouver, British Columbia, Canada, 2007.
[7] L. Bottou and V. Vapnik. Local learning algorithms. Neural Computation, 4(6):888–900, 1992.
[8] R. Caruana. Multitask learning. Machine Learning, 28(1):41–75, 1997.
[9] T. Evgeniou and M. Pontil. Regularized multi-task learning. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 109–117, Seattle, Washington, USA, 2004.
[10] T. V. Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De Moor, and J. Vandewalle. Benchmarking least squares support vector machine classifiers. Machine Learning, 54(1):5–32, 2004.
[11] M. Grant and S. Boyd. CVX: Matlab software for disciplined convex programming, 2011.
[12] L. Jacob, F. Bach, and J.-P. Vert. Clustered multi-task learning: a convex formulation. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 745–752, Vancouver, British Columbia, Canada, 2008.
[13] A. Kumar and H. Daum´ III. Learning task grouping and overlap in multi-task learning. In Proceedings e of the 29 th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012.
[14] S. Parameswaran and K. Weinberger. Large margin multi-task metric learning. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 1867–1875, 2010.
[15] J. C. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Sch¨ lkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods: Support Vector o Learning. MIT Press, 1998.
[16] S. Thrun. Is learning the n-th thing any easier than learning the first? In D. S. Touretzky, M. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 640–646, Denver, CO, 1995.
[17] S. Thrun and J. O’Sullivan. Discovering structure in multiple learning tasks: The TC algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 489–497, Bari, Italy, 1996.
[18] M. Wu and B. Sch¨ lkopf. A local learning approach for clustering. In B. Sch¨ lkopf, J. C. Platt, and o o T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 1529–1536, Vancouver, British Columbia, Canada, 2006.
[19] M. Wu, K. Yu, S. Yu, and B. Sch¨ lkopf. Local learning projections. In Proceedings of the Twenty-Fourth o International Conference on Machine Learning, pages 1039–1046, Corvallis, Oregon, USA, 2007.
[20] Y. Zhang and D.-Y. Yeung. A convex formulation for learning task relationships in multi-task learning. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, pages 733–742, Catalina Island, California, 2010. 9