nips nips2011 nips2011-291 nips2011-291-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Alessandro Lazaric, Marcello Restelli
Abstract: Transfer reinforcement learning (RL) methods leverage on the experience collected on a set of source tasks to speed-up RL algorithms. A simple and effective approach is to transfer samples from source tasks and include them in the training set used to solve a target task. In this paper, we investigate the theoretical properties of this transfer method and we introduce novel algorithms adapting the transfer process on the basis of the similarity between source and target tasks. Finally, we report illustrative experimental results in a continuous chain problem.
[1] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Vaughan. A theory of learning from different domains. Machine Learning, 79:151–175, 2010.
[2] Koby Crammer, Michael Kearns, and Jennifer Wortman. Learning from multiple sources. Journal of Machine Learning Research, 9:1757–1774, 2008.
[3] Damien Ernst, Pierre Geurts, and Louis Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503–556, 2005.
[4] M.G. Lagoudakis and R. Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107–1149, 2003.
[5] A. Lazaric, M. Restelli, and A. Bonarini. Transfer of samples in batch reinforcement learning. In Proceedings of the Twenty-Fifth Annual International Conference on Machine Learning (ICML’08), pages 544–551, 2008.
[6] Alessandro Lazaric. Knowledge Transfer in Reinforcement Learning. PhD thesis, Poltecnico di Milano, 2008.
[7] Alessandro Lazaric and Marcello Restelli. Transfer from Multiple MDPs. Technical Report 00618037, INRIA, 2011.
[8] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In Proceedings of the 22nd Conference on Learning Theory (COLT’09), 2009.
[9] R. Munos and Cs. Szepesv´ ri. Finite time bounds for fitted value iteration. Journal of Machine a Learning Research, 9:815–857, 2008.
[10] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
[11] Matthew E. Taylor, Nicholas K. Jong, and Peter Stone. Transferring instances for model-based reinforcement learning. In Proceedings of the European Conference on Machine Learning (ECML’08), pages 488–505, 2008.
[12] Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research, 10(1):1633–1685, 2009. 9