jmlr jmlr2005 jmlr2005-11 jmlr2005-11-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Andreas Maurer
Abstract: A mechnism of transfer learning is analysed, where samples drawn from different learning tasks of an environment are used to improve the learners performance on a new task. We give a general method to prove generalisation error bounds for such meta-algorithms. The method can be applied to the bias learning model of J. Baxter and to derive novel generalisation bounds for metaalgorithms searching spaces of uniformly stable algorithms. We also present an application to regularized least squares regression. Keywords: algorithmic stability, meta-learning, learning to learn
M. Anthony, P. Bartlett, Learning in Neural Networks: Theoretical Foundations, Cambridge University Press 1999. J. Baxter, Theoretical Models of Learning to Learn, in Learning to Learn, S. Thrun, L. Pratt Eds. Springer 1998. J. Baxter, A Model of Inductive Bias Learning, Journal of Artificial Intelligence Research 12: 149198, 2000. O. Bousquet, A. Elisseeff, “Stability and Generalization”, Journal of Machine Learning Research, 2: 499-526, 2002. R. Caruana, Multitask Learning, in Learning to Learn, S. Thrun, L. Pratt Eds. Springer 1998. N. Christianini, J. Shawe-Taylor, Support Vector Machines, Cambridge University Press 2000. L. Devroye, L. Gy¨ rfi, G. Lugosi, A Probabilistic Theory of Pattern Recognition. Springer, 1996. o S. Edelman, Representation, similarity and the chorus of prototypes. Minds and Machines, 45-68, 1995. W. Hoeffding, “Probability inequalities for sums of bounded random variables”, Journal of the American Statistical Association, 58:13-30, 1963. S. Kutin, P. Niyogi, Almost-everywhere algorithmic stability and generalization performance, Technical report , Department of Computer Science, University of Chicago, 2002. 993 M AURER D. McAllester, “Some PAC-Bayesian Theorems”, Proceedings of the Eleventh Annual Conference In Computational Learning Theory, 230-234, 1998. C. McDiarmid, “Concentration”, in Probabilistic Methods of Algorithmic Discrete Mathematics, p. 195-248. Springer, Berlin, 1998. A. Robins, Transfer in Congnition, in Learning to Learn, S. Thrun, L. Pratt Eds. Springer 1998. S. Thrun, Explanation-Based Neural Network Learning, Kluwer 1996. S.Thrun, Lifelong Learning Algorithms, in Learning to Learn, S.Thrun, L.Pratt Eds. Springer 1998. V. Vapnik, The Nature of Statistical Learning Theory, Springer 1995. D. H. Wolpert, The Mathematics of Generalization, Addison Wesley, 1995. 994