nips nips2003 nips2003-141 nips2003-141-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Christopher J. Paciorek, Mark J. Schervish
Abstract: We introduce a class of nonstationary covariance functions for Gaussian process (GP) regression. Nonstationary covariance functions allow the model to adapt to functions whose smoothness varies with the inputs. The class includes a nonstationary version of the Matérn stationary covariance, in which the differentiability of the regression function is controlled by a parameter, freeing one from fixing the differentiability in advance. In experiments, the nonstationary GP regression model performs well when the input space is two or three dimensions, outperforming a neural network model and Bayesian free-knot spline models, and competitive with a Bayesian neural network, but is outperformed in one dimension by a state-of-the-art Bayesian free-knot spline model. The model readily generalizes to non-Gaussian data. Use of computational methods for speeding GP fitting may allow for implementation of the method on larger datasets. 1
[1] M.N. Gibbs. Bayesian Gaussian Processes for Classification and Regression. PhD thesis, Univ. of Cambridge, Cambridge, U.K., 1997.
[2] D.J.C. MacKay. Introduction to Gaussian processes. Technical report, Univ. of Cambridge, 1997.
[3] D. Higdon, J. Swall, and J. Kern. Non-stationary spatial modeling. In J.M. Bernardo, J.O. Berger, A.P. Dawid, and A.F.M. Smith, editors, Bayesian Statistics 6, pages 761–768, Oxford, U.K., 1999. Oxford University Press.
[4] A.M. Schmidt and A. O’Hagan. Bayesian inference for nonstationary spatial covariance structure via spatial deformations. Technical Report 498/00, University of Sheffield, 2000.
[5] D. Damian, P.D. Sampson, and P. Guttorp. Bayesian estimation of semi-parametric nonstationary spatial covariance structure. Environmetrics, 12:161–178, 2001.
[6] I. DiMatteo, C.R. Genovese, and R.E. Kass. Bayesian curve-fitting with free-knot splines. Biometrika, 88:1055–1071, 2002.
[7] D. MacKay and R. Takeuchi. Interpolation models with multiple hyperparameters, 1995.
[8] Volker Tresp. Mixtures of Gaussian processes. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13, pages 654–660. MIT Press, 2001.
[9] C.E. Rasmussen and Z. Ghahramani. Infinite mixtures of Gaussian process experts. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, Massachusetts, 2002. MIT Press.
[10] C.J. Paciorek. Nonstationary Gaussian Processes for Regression and Spatial Modelling. PhD thesis, Carnegie Mellon University, Pittsburgh, Pennsylvania, 2003.
[11] M.L. Stein. Interpolation of Spatial Data : Some Theory for Kriging. Springer, N.Y., 1999.
[12] F. Vivarelli and C.K.I. Williams. Discovering hidden features with Gaussian processes regression. In M.J. Kearns, S.A. Solla, and D.A. Cohn, editors, Advances in Neural Information Processing Systems 11, 1999.
[13] J.R. Lockwood, M.J. Schervish, P.L. Gurian, and M.J. Small. Characterization of arsenic occurrence in source waters of U.S. community water systems. J. Am. Stat. Assoc., 96:1184–1193, 2001.
[14] C.C. Holmes and B.K. Mallick. Bayesian regression with multivariate linear splines. Journal of the Royal Statistical Society, Series B, 63:3–17, 2001.
[15] D.G.T. Denison, B.K. Mallick, and A.F.M. Smith. Bayesian MARS. Statistics and Computing, 8:337–346, 1998.
[16] J.H. Friedman. Multivariate adaptive regression splines. Annals of Statistics, 19:1–141, 1991.
[17] S.A. Wood, W.X. Jiang, and M. Tanner. Bayesian mixture of splines for spatially adaptive nonparametric regression. Biometrika, 89:513–528, 2002.
[18] S.M. Bruntz, W.S. Cleveland, B. Kleiner, and J.L. Warner. The dependence of ambient ozone on solar radiation, temperature, and mixing height. In American Meteorological Society, editor, Symposium on Atmospheric Diffusion and Air Pollution, pages 125–128, 1974.
[19] C. Biller. Adaptive Bayesian regression splines in semiparametric generalized linear models. Journal of Computational and Graphical Statistics, 9:122–140, 2000.
[20] A.J. Smola and P. Bartlett. Sparse greedy Gaussian process approximation. In T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, Cambridge, Massachusetts, 2001. MIT Press.
[21] M. Seeger and C. Williams. Fast forward selection to speed up sparse Gaussian process regression. In Workshop on AI and Statistics 9, 2003.