jmlr jmlr2010 jmlr2010-104 jmlr2010-104-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Miguel Lázaro-Gredilla, Joaquin Quiñonero-Candela, Carl Edward Rasmussen, Aníbal R. Figueiras-Vidal
Abstract: We present a new sparse Gaussian Process (GP) model for regression. The key novel idea is to sparsify the spectral representation of the GP. This leads to a simple, practical algorithm for regression tasks. We compare the achievable trade-offs between predictive accuracy and computational requirements, and show that these are typically superior to existing state-of-the-art sparse approximations. We discuss both the weight space and function space representations, and note that the new construction implies priors over functions which are always stationary, and can approximate any covariance function in this class. Keywords: Gaussian process, probabilistic regression, sparse approximation, power spectrum, computational efficiency
G. L. Bretthorst. Nonuniform sampling: Bandwidth and aliasing. In Maximum Entropy and Bayesian Methods, pages 1–28. Kluwer, 2000. A. B. Carlson. Communication Systems. McGraw-Hill, 3rd edition, 1986. L. Csató and M. Opper. Sparse online Gaussian processes. Neural Computation, 14(3):641–669, 2002. M. Lázaro-Gredilla. Sparse Gaussian Processes for Large-Scale Machine Learning. PhD thesis, Universidad Carlos III de Madrid, 2010. URL http://www.tsc.uc3m.es/~miguel/ publications.php. M. Lázaro-Gredilla and A.R. Figueiras-Vidal. Inter-domain Gaussian processes for sparse inference using inducing features. In Advances in Neural Information Processing Systems 22, pages 1087– 1095. MIT Press, 2010. M. Lázaro-Gredilla, J. Quiñonero-Candela, and A. Figueiras-Vidal. Sparse spectral sampling Gaussian processes. Technical report, Microsoft Research, 2007. D. J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003. 1880 S PARSE S PECTRUM G AUSSIAN P ROCESS R EGRESSION A. Naish-Guzman and S. Holden. The generalized FITC approximation. In Advances in Neural Information Processing Systems 20, pages 1057–1064. MIT Press, 2008. J. Quiñonero-Candela and C. E. Rasmussen. A unifying view of sparse approximate Gaussian process regression. Journal of Machine Learning Research, 6:1939–1959, 2005. J. Quiñonero-Candela, C. E. Rasmussen, and C. K. I. Williams. Approximation methods for Gaussian process regression. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, LargeScale Kernel Machines, pages 203–223. MIT Press, 2007. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems 20, pages 1177–1184. MIT Press, Cambridge, MA, 2008. C. E. Rasmussen and Joaquin Quiñonero-Candela. Healing the relevance vector machine through augmentation. In Proceedings of the 22nd International Conference on Machine Learning, pages 689–696, 2005. C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006. M. Seeger, C. K. I. Williams, and N. D. Lawrence. Fast forward selection to speed up sparse Gaussian process regression. In Proceedings of the 9th International Workshop on AI Stats, 2003. A. J. Smola and P. Bartlett. Sparse greedy Gaussian process regression. In Advances in Neural Information Processing Systems 13, pages 619–625. MIT Press, 2001. E. Snelson and Z. Ghahramani. Sparse Gaussian processes using pseudo-inputs. In Advances in ˝ Neural Information Processing Systems 18, pages 1259U–1266. MIT Press, 2006. M. L. Stein. Interpolation of Spatial Data. Springer Verlag, 1999. M. K. Titsias. Variational learning of inducing variables in sparse Gaussian processes. In Proceedings of the 12th International Workshop on AI Stats, 2009. V. Tresp. A Bayesian committee machine. Neural Computation, 12:2719–2741, 2000. R. Urtasun and T. Darrell. Sparse probabilistic regression for activity-independent human pose inference. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, pages 1–8, 2008. C. Walder, K. I. Kim, and B. Schölkopf. Sparse multiscale Gaussian process regression. In 25th International Conference on Machine Learning. ACM Press, New York, 2008. C. K. I. Williams. Computing with infinite networks. In Advances in Neural Information Processing Systems 9, pages 1069–1072. MIT Press, 1997. C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13, pages 682–688. MIT Press, 2001. 1881