nips nips2005 nips2005-113 nips2005-113-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jian Zhang, Zoubin Ghahramani, Yiming Yang
Abstract: We propose a probabilistic model based on Independent Component Analysis for learning multiple related tasks. In our model the task parameters are assumed to be generated from independent sources which account for the relatedness of the tasks. We use Laplace distributions to model hidden sources which makes it possible to identify the hidden, independent components instead of just modeling correlations. Furthermore, our model enjoys a sparsity property which makes it both parsimonious and robust. We also propose efficient algorithms for both empirical Bayes method and point estimation. Our experimental results on two multi-label text classification data sets show that the proposed approach is promising.
[1] Ando, R. and Zhang, T. A Framework for Learning Predicative Structures from Multiple Tasks and Unlabeled Data. Technical Rerport RC23462, IBM T.J. Watson Research Center, 2004.
[2] Baxter, J. A Model for Inductive Bias Learning. J. of Artificial Intelligence Research, 2000.
[3] Breiman, L. and Friedman J. Predicting Multivariate Responses in Multiple Linear Regression. J. Royal Stat. Society B, 59:3-37, 1997.
[4] Evgeniou, T., Micchelli, C. and Pontil, M. Learning Multiple Tasks with Kernel Methods. J. of Machine Learning Research, 6:615-637, 2005.
[5] Ghosn, J. and Bengio, Y. Bias Learning, Knowledge Sharing. IEEE Transaction on Neural Networks, 14(4):748-765, 2003.
[6] Heskes, T. Empirical Bayes for Learning to Learn. In Proc. of the 17th ICML, 2000.
[7] Jaakkola, T. and Jordan, M. A Variational Approach to Bayesian Logistic Regression Models and Their Extensions. In Proc. of the Sixth Int. Workshop on AI and Statistics, 1997.
[8] Koller, D. and Sahami, M. Hierarchically Classifying Documents using Very Few Words. In Proc. of the 14th ICML, 1997.
[9] Roberts, S. and Everson, R. (editors). Independent Component Analysis: Principles and Practice, Cambridge University Press, 2001.
[10] Teh, Y.-W., Seeger, M. and Jordan, M. Semiparametric Latent Factor Models. In Z. Ghahramani and R. Cowell, editors, Workshop on Artificial Intelligence and Statistics 10, 2005.
[11] Yu, K., Tresp, V. and Schwaighofer, A. Learning Gaussian Processes from Multiple Tasks. In Proc. of the 22nd ICML, 2005.