nips nips2010 nips2010-242 nips2010-242-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Iain Murray, Ryan P. Adams
Abstract: The Gaussian process (GP) is a popular way to specify dependencies between random variables in a probabilistic model. In the Bayesian framework the covariance structure can be specified using unknown hyperparameters. Integrating over these hyperparameters considers different possible explanations for the data when making predictions. This integration is often performed using Markov chain Monte Carlo (MCMC) sampling. However, with non-Gaussian observations standard hyperparameter sampling approaches require careful tuning and may converge slowly. In this paper we present a slice sampling approach that requires little tuning while mixing well in both strong- and weak-data regimes. 1
[1] Iain Murray, Ryan Prescott Adams, and David J.C. MacKay. Elliptical slice sampling. Journal of Machine Learning Research: W&CP;, 9:541–548, 2010. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS).
[2] Radford M. Neal. Slice sampling. Annals of Statistics, 31(3):705–767, 2003.
[3] Deepak K. Agarwal and Alan E. Gelfand. Slice sampling for simulation based fitting of spatial data models. Statistics and Computing, 15(1):61–69, 2005.
[4] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for machine learning. MIT Press, 2006.
[5] Luke Tierney. Markov chains for exploring posterior distributions. The Annals of Statistics, 22(4):1701–1728, 1994.
[6] Michalis Titsias, Neil D Lawrence, and Magnus Rattray. Efficient sampling for Gaussian process inference using control variables. In Advances in Neural Information Processing Systems 21, pages 1681–1688. MIT Press, 2009.
[7] Simon Duane, A. D. Kennedy, Brian J. Pendleton, and Duncan Roweth. Hybrid Monte Carlo. Physics Letters B, 195(2):216–222, September 1987.
[8] Radford M. Neal. MCMC using Hamiltonian dynamics. To appear in the Handbook of Markov Chain Monte Carlo, Chapman & Hall / CRC Press, 2011. http://www.cs.toronto.edu/~radford/ftp/ham-mcmc.pdf.
[9] Ole F. Christensen, Gareth O. Roberts, and Martin Sk˜ld. Robust Markov chain Monte a Carlo methods for spatial generalized linear mixed models. Journal of Computational and Graphical Statistics, 15(1):1–17, 2006.
[10] Thomas Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Annual Conference on Uncertainty in Artificial Intelligence (UAI), pages 362–369, 2001. Corrected version available from http://research.microsoft.com/~minka/papers/ep/.
[11] Kiam Choo. Learning hyperparameters for neural network models using Hamiltonian dynamics. Master’s thesis, Department of Computer Science, University of Toronto, 2000. Available from http://www.cs.toronto.edu/~radford/ftp/kiam-thesis.ps.
[12] Mary Kathryn Cowles, Nicky Best, Karen Vines, and Martyn Plummer. R-CODA 0.10-5, 2006. http://www-fis.iarc.fr/coda/.
[13] Michalis Titsias. Auxiliary sampling using imaginary data, 2010. Unpublished.
[14] Radford M. Neal. Regression and classification using Gaussian process priors. In J. M. Bernardo et al., editors, Bayesian Statistics 6, pages 475–501. OU Press, 1999.
[15] Malte Kuss and Carl Edward Rasmussen. Assessing approximate inference for binary Gaussian process classification. Journal of Machine Learning Research, 6:1679–1704, 2005.
[16] V. G. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker. Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Technical Digest, 10:262–266, 1989.
[17] R. G. Jarrett. A note on the intervals between coal-mining disasters. Biometrika, 66 (1):191–193, 1979.
[18] Brian D. Ripley. Modelling spatial patterns. Journal of the Royal Statistical Society, Series B, 39:172–212, 1977.
[19] Mark Girolami and Ben Calderhead. Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society. Series B (Methodological), 2011. To appear. 9