nips nips2002 nips2002-41 nips2002-41-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zoubin Ghahramani, Carl E. Rasmussen
Abstract: We investigate Bayesian alternatives to classical Monte Carlo methods for evaluating integrals. Bayesian Monte Carlo (BMC) allows the incorporation of prior knowledge, such as smoothness of the integrand, into the estimation. In a simple problem we show that this outperforms any classical importance sampling method. We also attempt more challenging multidimensional integrals involved in computing marginal likelihoods of statistical models (a.k.a. partition functions and model evidences). We find that Bayesian Monte Carlo outperformed Annealed Importance Sampling, although for very high dimensional problems or problems with massive multimodality BMC may be less adequate. One advantage of the Bayesian approach to Monte Carlo is that samples can be drawn from any distribution. This allows for the possibility of active design of sample points so as to maximise information gain.
Friedman, J. (1988). Multivariate Adaptive Regression Splines. Technical Report No. 102, November 1988, Laboratory for Computational Statistics, Department of Statistics, Stanford University. Kennedy, M. (1998). Bayesian quadrature with non-normal approximating functions, Statistics and Computing, 8, pp. 365–375. MacKay, D. J. C. (1999). Introduction to Monte Carlo methods. In Learning in Graphical Models, M. I. Jordan (ed), MIT Press, 1999. Gelman, A. and Meng, X.-L. (1998) Simulating normalizing constants: From importance sampling to bridge sampling to path sampling, Statistical Science, vol. 13, pp. 163–185. Minka, T. P. (2000) Deriving quadrature rules from Gaussian processes, Technical Report, Statistics Department, Carnegie Mellon University. Neal, R. M. (2001). Annealed Importance Sampling, Statistics and Computing, 11, pp. 125–139. O’Hagan, A. (1987). Monte Carlo is fundamentally unsound, The Statistician, 36, pp. 247-249. O’Hagan, A. (1991). Bayes-Hermite Quadrature, Journal of Statistical Planning and Inference, 29, pp. 245–260. O’Hagan, A. (1992). Some Bayesian Numerical Analysis. Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds), Oxford University Press, pp. 345–365 (with discussion). C. E. Rasmussen (2003). Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals, Bayesian Statistics 7 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds), Oxford University Press. Williams, C. K. I. and C. E. Rasmussen (1996). Gaussian Processes for Regression, in D. S. Touretzky, M. C. Mozer and M. E. Hasselmo (editors), NIPS 8, MIT Press.