nips nips2010 nips2010-146 nips2010-146-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Arvind Agarwal, Samuel Gerber, Hal Daume
Abstract: We present a novel method for multitask learning (MTL) based on manifold regularization: assume that all task parameters lie on a manifold. This is the generalization of a common assumption made in the existing literature: task parameters share a common linear subspace. One proposed method uses the projection distance from the manifold to regularize the task parameters. The manifold structure and the task parameters are learned using an alternating optimization framework. When the manifold structure is fixed, our method decomposes across tasks which can be learnt independently. An approximation of the manifold regularization scheme is presented that preserves the convexity of the single task learning problem, and makes the proposed MTL framework efficient and easy to implement. We show the efficacy of our method on several datasets. 1
[1] A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. In NIPS ’06, 2006.
[2] A. Argyriou, T. Evgeniou, M. Pontil, A. Argyriou, T. Evgeniou, and M. Pontil. Convex multitask feature learning. In Machine Learning. press, 2007.
[3] A. Argyriou, C. A. Micchelli, and M. Pontil. When is there a representer theorem? vector versus matrix regularizers. J. Mach. Learn. Res., 10:2507–2529, 2009.
[4] A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying. A spectral regularization framework for multi-task structure learning. In NIPS ’08. 2008.
[5] B. Bakker and T. Heskes. Task clustering and gating for bayesian multitask learning. JMLR, 4:2003, 2003.
[6] M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15:1373–1396, 2002.
[7] M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res., 7:2399–2434, 2006.
[8] R. Caruana. Multitask learning. In Machine Learning, pages 41–75, 1997.
[9] H. Daum´ III. Bayesian multitask learning with latent hierarchies. In Conference on Uncere tainty in Artificial Intelligence ’09, Montreal, Canada, 2009.
[10] T. Evgeniou, C. A. Micchelli, and M. Pontil. Learning multiple tasks with kernel methods. JMLR, 6:615–637, 2005.
[11] S. Gerber, T. Tasdizen, and R. Whitaker. Dimensionality reduction and principal surfaces via kernel map manifolds. In In Proceedings of the 2009 International Conference on Computer Vison (ICCV), 2009.
[12] T. Hastie. Principal curves and surfaces. PhD thesis, Stanford University, 1984.
[13] L. Jacob, F. Bach, and J.-P. Vert. Clustered multi-task learning: A convex formulation. In NIPS ’08, 2008.
[14] P. J. Lenk, W. S. DeSarbo, P. E. Green, and M. R. Young. Hierarchical bayes conjoint analysis: Recovery of partworth heterogeneity from reduced experimental designs. MARKETING SCIENCE, 1996.
[15] Q. Liu, X. Liao, H. L. Carin, J. R. Stack, and L. Carin. Semisupervised multitask learning. IEEE 2009, 2009.
[16] C. A. Micchelli and M. Pontil. Regularized multi-task learning. In KDD 2004, pages 109–117, 2004.
[17] S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, December 2000.
[18] J. B. Tenenbaum, V. Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, December 2000.
[19] S. Thrun and L. Pratt, editors. Learning to learn. Kluwer Academic Publishers, Norwell, MA, USA, 1998.
[20] K. Q. Weinberger, F. Sha, and L. K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction. In In ICML 2004, pages 839–846. ACM Press, 2004.
[21] Y. Xue, X. Liao, L. Carin, and B. Krishnapuram. Multi-task learning for classification with dirichlet process priors. J. Mach. Learn. Res., 8:35–63, 2007.
[22] K. Yu, V. Tresp, and A. Schwaighofer. Learning gaussian processes from multiple tasks. In ICML ’05, 2005.
[23] J. Zhang, Z. Ghahramani, and Y. Yang. Flexible latent variable models for multi-task learning. Mach. Learn., 73(3):221–242, 2008.
[24] J. Zhang, J. Zhang, Y. Yang, Z. Ghahramani, and Y. Yang. Learning multiple related tasks using latent independent component analysis. In NIPS ’05, 2005. 9