nips nips2007 nips2007-170 nips2007-170-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Andrew Naish-guzman, Sean Holden
Abstract: We propose a Gaussian process (GP) framework for robust inference in which a GP prior on the mixing weights of a two-component noise model augments the standard process over latent function values. This approach is a generalization of the mixture likelihood used in traditional robust GP regression, and a specialization of the GP mixture models suggested by Tresp [1] and Rasmussen and Ghahramani [2]. The value of this restriction is in its tractable expectation propagation updates, which allow for faster inference and model selection, and better convergence than the standard mixture. An additional benefit over the latter method lies in our ability to incorporate knowledge of the noise domain to influence predictions, and to recover with the predictive distribution information about the outlier distribution via the gating process. The model has asymptotic complexity equal to that of conventional robust methods, but yields more confident predictions on benchmark problems than classical heavy-tailed models and exhibits improved stability for data with clustered corruptions, for which they fail altogether. We show further how our approach can be used without adjustment for more smoothly heteroscedastic data, and suggest how it could be extended to more general noise models. We also address similarities with the work of Goldberg et al. [3].
[1] Volker Tresp. Mixtures of Gaussian processes. In Advances in Neural Information Processing Systems, pages 654–660, 2000.
[2] Carl Edward Rasmussen and Zoubin Ghahramani. Infinite mixtures of gaussian process experts. In Advances in Neural Information Processing Systems, 2002.
[3] Paul Goldberg, Christopher Williams, and Christopher Bishop. Regression with input-dependent noise: a Gaussian process treatment. In Advances in Neural Information Processing Systems. MIT Press, 1998.
[4] Edward Snelson and Zoubin Ghahramani. Sparse Gaussian processes using pseudo-inputs. In Advances in Neural Information Processing Systems 18. MIT Press, 2005.
[5] Thomas Minka. A family of algorithms for approximate Bayesian inference. PhD thesis, Massachusetts Institute of Technology, 2001.
[6] Carl Rasmussen and Christopher Williams. Gaussian processes for machine learning. MIT Press, 2006.
[7] Malte Kuss. Gaussian process models for robust regression, classification and reinforcement learning. PhD thesis, Technische Universit¨ t Darmstadt, 2006. a
[8] Matthias Seeger. Expectation propagation for exponential families, 2005. Available from http://www.cs.berkeley.edu/˜mseeger/papers/epexpfam.ps.gz.
[9] J. H. Friedman. Multivariate adaptive regression splines. Annals of Statistics, 19(1):1–67, 1991.
[10] B.W. Silverman. Some aspects of the spline smoothing approach to non-parametric regression curve fitting. Journal of the Royal Statistical Society B, 47:1–52, 1985.
[11] Edward Snelson, Carl Edward Rasmussen, and Zoubin Ghahramani. Warped Gaussian processes. In Advances in Neural Information Processing Systems 16, 2003. 8