nips nips2001 nips2001-95 nips2001-95-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Carl E. Rasmussen, Zoubin Ghahramani
Abstract: We present an extension to the Mixture of Experts (ME) model, where the individual experts are Gaussian Process (GP) regression models. Using an input-dependent adaptation of the Dirichlet Process, we implement a gating network for an infinite number of Experts. Inference in this model may be done efficiently using a Markov Chain relying on Gibbs sampling. The model allows the effective covariance function to vary with the inputs, and may handle large datasets – thus potentially overcoming two of the biggest hurdles with GP models. Simulations show the viability of this approach.
Gibbs, M. N. (1997). Bayesian Gaussian Processes for Regression and Classification. PhD thesis. University of Cambridge. Goldberg, P. W., Williams, C. K. I., & Bishop C. M. (1998). Regression with Inputdependent Noise, NIPS 10. Duane, S., Kennedy, A. D., Pendleton, B. J., and Roweth, D. (1987). Hybrid Monte Carlo, Physics letters B, vol. 55, pp. 2774–2777. Gilks, W. R. & Wild, P. (1992). Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41, 337–348. Jacobs, R. A., Jordan, M. I., Nowlan, S. J. & Hinton, G. E. (1991). Adaptive mixture of local experts. Neural Computation, vol 3, pp 79–87. Neal, R. M. (1998). Markov chain sampling methods for Dirichlet process mixture models. Technical Report 4915, Department of Statistics, University of Toronto. http://www.cs.toronto.edu/ radford/mixmc.abstract.html. Rasmussen, C. E. (2000). The Infinite Gaussian Mixture Model, NIPS 12, S.A. Solla, T.K. Leen and K.-R. M¨ ller (eds.), pp. 554–560, MIT Press. u Silverman, B. W. (1985). Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. Royal Stat. Society. Ser. B, vol. 47, pp. 1–52. Smola A. J. and Bartlett, P. (2001). Sparse Greedy Gaussian Process Regression, NIPS 13. Tresp V. (2001). Mixtures of Gaussian Process, NIPS 13. Williams, C. K. I. and Seeger, M. (2001). Using the Nystr¨ m Method to Speed Up Kernel o Machines, NIPS 13. Williams, C. K. I. and C. E. Rasmussen (1996). Gaussian Processes for Regression, in D. S. Touretzky, M. C. Mozer and M. E. Hasselmo (editors), NIPS 8, MIT Press.