jmlr jmlr2009 jmlr2009-88 jmlr2009-88-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Leslie Foster, Alex Waagen, Nabeela Aijaz, Michael Hurley, Apolonio Luis, Joel Rinsky, Chandrika Satyavolu, Michael J. Way, Paul Gazis, Ashok Srivastava
Abstract: The use of Gaussian processes can be an effective approach to prediction in a supervised learning environment. For large data sets, the standard Gaussian process approach requires solving very large systems of linear equations and approximations are required for the calculations to be practical. We will focus on the subset of regressors approximation technique. We will demonstrate that there can be numerical instabilities in a well known implementation of the technique. We discuss alternate implementations that have better numerical stability properties and can lead to better predictions. Our results will be illustrated by looking at an application involving prediction of galaxy redshift from broadband spectrum data. Keywords: Gaussian processes, low rank approximations, numerical stability, photometric redshift, subset of regressors method
Ed Anderson, Zhaojun Bai, Christian H. Bischof, Susan Blackford, James W. Demmel, Jack J. Dongarra, Jeremy J. Du Croz, Anne Greenbaum, Sven J. Hammarling, Alan McKenney, and Danny C. Sorensen. LAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, third edition, 1999. ISBN 0-89871-447-8. ˚ Ake Bj¨ rck. Numerical Methods for Least Squares Problems. Society for Industrial and Applied o Mathematics, Philadelphia, PA, USA, 1996. ISBN 0-89871-360-9. Bem Cayco, Wasin So, Miranda Braselton, Kelley Cartwright, Michael Hurley, Maheen Khan, Miguel Rodriguez, David Shao, Jason Smith, Jimmy Ying, and Genti Zaimi. Camcos project – Fall 2006: Improved linear algebra methods for redshift computation from limited spectrum data. At www.math.sjsu.edu/˜foster/camcos07/redshift.html, 2006. Lehel Csato and Manfred Opper. Sparse on-line gaussian processes. Neural Computation, 14: 641–668, 2002. Jack J. Dongarra, James R. Bunch, Cleve B. Moler, and G. W. Stewart. LINPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1979. ISBN 0-89871172-X. Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootsrap. Chapman and Hall, New York, 1993. Shai Fine and Katya Scheinberg. Efficient svm training using low-rank kernel representations. J. of Machine Learning Research, 2:243–264, 2001. George E. Forsythe and Cleve B. Moler. Computer Solution of Linear Algebraic Systems. PrenticeHall, Englewood Cliffs, NJ, USA, 1967. Leslie Foster, Alex Waagen, Nabeela Aijaz, Michael Hurley, Apolo Luis, Joel Rinsky, Chandrika Satyavolu, Ashok Srivastava, Paul Gazis, and Michael Way. Improved linear algebra methods for redshift computation from limited spectrum data - II. NASA Technical Report NASA/TM-2008214571, NASA Ames Research Center, Moffett Field, CA, 2008. Available at ntrs.nasa.gov and at www.math.sjsu.edu/˜foster/camcos07/redshift.html. Gene H. Golub and Charles F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, USA, third edition, 1996. ISBN 0-8018-5413-X, 0-8018-5414-8. Ming Gu and Stanley C. Eisenstat. Efficient algorithm for computing a strong rank-revealing qr factorization. SIAM J. Sci. Comput., 17(4):848–869, 1996. Ming Gu and Luiza Miranian. Strong rank-revealing Cholesky factorization. Electronic Transactions on Numerical Analysis, 17:76–92, 2004. Per Christian Hansen. Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia, PA, USA, 1998. 880 S TABLE AND E FFICIENT G AUSSIAN P ROCESS C ALCULATIONS Nicholas J. Higham. Analysis of the Cholesky decomposition of a semi-definite matrix. In M. G. Cox and S. J. Hammarling, editors, Reliable Numerical Computation, pages 161–185. Oxford University Press, 1990. Nicholas J. Higham. Accuracy and Stability of Numerical Algorithms. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, second edition, 2002. ISBN 0-89871-521-0. Craig Lucas. LAPACK-style codes for level 2 and 3 pivoted cholesky factorizations. Numerical Analysis Report No. 442, Manchester Centre for Computational Mathematics, Manchester, England, 2004. LAPACK Working Note 161. Douglas C. Montgomery, Elizabeth A. Peck, and G. Geoffrey Vining. Introduction to Linear Regression Analysis. John Wiley and Sons, Hoboken, NJ, USA, fourth edition, 2006. Gwen Peters and James H. Wilkinson. The least squares problem and pseudo-inverses. Comput. J., 13(3):309–316, 1970. Tomaso Poggio and Fedirico Girosi. Networks for approximation and learning. Proceedings of IEEE, 78:1481–1497, 1990. Joaquin Quinonero-Candela and Carl E. Rasmussen. A unifying view of sparse approximate Gaussian process regression. J. of Machine Learning Research, 6:1939–1959, 2005. Carl E. Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachusetts, 2006. Matthias Seeger, Christopher Williams, and Neil D. Lawrence. Fast forward selection to speed up sparse gaussian process regression. In C. M. Bishop and B. J. Frey, editors, Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, San Francisco, 2003. Morgan Kaufmann. Alex J. Smola and Peter Bartlett. Sparse greedy gaussian process regression. In T. Leen, T. Diettrich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 619–625. MIT Press, 2001. Edward Snelson and Zoubin Ghahramani. Sparse gaussian process using pseudo-inputs. In Y. Weiss, B. Scholkpf, and J. Platt, editors, Advances in Neural Information Processing Systems 18, pages 1257–1264. MIT Press, 2006. G. W. Stewart. The efficient generation of random orthogonal matrices with an application to condition estimators. SIAM J. Numer. Anal., 17(3):403–409, 1980. Lloyd N. Trefethen and David Bau III. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1997. ISBN 0-89871-361-7. Sethu Vijayakumar, Aaron DSouza, Tomohiro Shibata, Jorg Conradt, and Stefan Schaal. Statistical learning for humanoid robots. Autonomous Robot, 12:55–69, 2002. Grace Wahba. Spline Models for Observation Data. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1990. 881 F OSTER , WAAGEN , A IJAZ , H URLEY, L UIS , R INSKY, S ATYAVOLU , WAY, G AZIS AND S RIVASTAVA Michael J. Way and Ashok Srivastava. Novel methods for predicting photometric redshifts from broadband data using virtual sensors. The Astrophysical Journal, 647:102–115, 2006. Donald G. York, J. Adelman, and John E. Anderson et. al. The sloan digital sky survey: Technical summary. The Astronomical Journal, 120:1579–1587, 2000. 882