nips nips2012 nips2012-33 nips2012-33-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Michael Osborne, Roman Garnett, Zoubin Ghahramani, David K. Duvenaud, Stephen J. Roberts, Carl E. Rasmussen
Abstract: Numerical integration is a key component of many problems in scientific computing, statistical modelling, and machine learning. Bayesian Quadrature is a modelbased method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normalising constant of a probabilistic model. Our approach approximately marginalises the quadrature model’s hyperparameters in closed form, and introduces an active learning scheme to optimally select function evaluations, as opposed to using Monte Carlo samples. We demonstrate our method on both a number of synthetic benchmarks and a real scientific problem from astronomy. 1
[1] R.M. Neal. Annealed importance sampling. Statistics and Computing, 11(2):125–139, 2001.
[2] J. Skilling. Nested sampling. Bayesian inference and maximum entropy methods in science and engineering, 735:395–405, 2004.
[3] M.H. Chen, Q.M. Shao, and J.G. Ibrahim. Monte Carlo methods in Bayesian computation. Springer, 2000.
[4] R. M. Neal. Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRGTR-93-1, University of Toronto, 1993.
[5] S.P. Brooks and G.O. Roberts. Convergence assessment techniques for Markov chain Monte Carlo. Statistics and Computing, 8(4):319–335, 1998.
[6] M.K. Cowles, G.O. Roberts, and J.S. Rosenthal. Possible biases induced by MCMC convergence diagnostics. Journal of Statistical Computation and Simulation, 64(1):87, 1999.
[7] P. Diaconis. Bayesian numerical analysis. In S. Gupta J. Berger, editor, Statistical Decision Theory and Related Topics IV, volume 1, pages 163–175. Springer-Verlag, New York, 1988.
[8] A. O’Hagan. Bayes-Hermite quadrature. Journal of Statistical Planning and Inference, 29:245–260, 1991.
[9] M. Kennedy. Bayesian quadrature with non-normal approximating functions. Statistics and Computing, 8(4):365–375, 1998.
[10] C. E. Rasmussen and Z. Ghahramani. Bayesian Monte Carlo. In S. Becker and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, Cambridge, MA, 2003.
[11] M.A. Osborne, R. Garnett, S.J. Roberts, C. Hart, S. Aigrain, N.P. Gibson, and S. Aigrain. Bayesian quadrature for ratios. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2012), 2012.
[12] C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
[13] T. P. Minka. Deriving quadrature rules from Gaussian processes. Technical report, Statistics Department, Carnegie Mellon University, 2000.
[14] R. Garnett, M.A. Osborne, S. Reece, A. Rogers, and S.J. Roberts. Sequential bayesian prediction in the presence of changepoints and faults. The Computer Journal, 53(9):1430, 2010.
[15] Sloan Digital Sky Survey, 2011. http://www.sdss.org/. 9