nips nips2010 nips2010-113 nips2010-113-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Fabian L. Wauthier, Michael I. Jordan
Abstract: Heavy-tailed distributions are often used to enhance the robustness of regression and classification methods to outliers in output space. Often, however, we are confronted with “outliers” in input space, which are isolated observations in sparsely populated regions. We show that heavy-tailed stochastic processes (which we construct from Gaussian processes via a copula), can be used to improve robustness of regression and classification estimators to such outliers by selectively shrinking them more strongly in sparse regions than in dense regions. We carry out a theoretical analysis to show that selective shrinkage occurs when the marginals of the heavy-tailed process have sufficiently heavy tails. The analysis is complemented by experiments on biological data which indicate significant improvements of estimates in sparse regions while producing competitive results in dense regions. 1
[1] Tamara Broderick and Robert B. Gramacy. Classification and Categorical Inputs with Treed Gaussian Process Models. Journal of Classification. To appear.
[2] Wei Chu and Zoubin Ghahramani. Gaussian Processes for Ordinal Regression. Journal of Machine Learning Research, 6:1019–1041, 2005.
[3] Doris Damian, Paul D. Sampson, and Peter Guttorp. Bayesian Estimation of Semi-Parametric Non-Stationary Spatial Covariance Structures. Environmetrics, 12:161–178.
[4] Adrian Dobra and Alex Lenkoski. Copula Gaussian Graphical Models. Technical report, Department of Statistics, University of Washington, 2009.
[5] Paul W. Goldberg, Christopher K. I. Williams, and Christopher M. Bishop. Regression with Input-dependent Noise: A Gaussian Process Treatment. In Advances in Neural Information Processing Systems, volume 10, pages 493–499. MIT Press, 1998.
[6] Robert B. Gramacy and Herbert K. H. Lee. Bayesian Treed Gaussian Process Models with an Application to Computer Modeling. Journal of the American Statistical Association, 2007.
[7] Sebastian Jaimungal and Eddie K. Ng. Kernel-based Copula Processes. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, pages 628–643. Springer-Verlag, 2009.
[8] David X. Li. On Default Correlation: A Copula Function Approach. Technical Report 99-07, Riskmetrics Group, New York, April 2000.
[9] Han Liu, John Lafferty, and Larry Wasserman. The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs. Journal of Machine Learning Research, 10:1–37, 2009.
[10] Roger B. Nelsen. An Introduction to Copulas. Springer, 1999.
[11] Michael Pitt, David Chan, and Robert J. Kohn. Efficient Bayesian Inference for Gaussian Copula Regression Models. Biometrika, 93(3):537–554, 2006.
[12] Carl E. Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
[13] Alexandra M. Schmidt and Anthony O’Hagan. Bayesian Inference for Nonstationary Spatial Covariance Structure via Spatial Deformations. Journal of the Royal Statistical Society, 65(3):743–758, 2003. Ser. B.
[14] John Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
[15] Ed Snelson, Carl E. Rasmussen, and Zoubin Ghahramani. Warped Gaussian Processes. In Advances in Neural Information Processing Systems, volume 16, pages 337–344, 2004.
[16] Peter Xue-Kun Song. Multivariate Dispersion Models Generated From Gaussian Copula. Scandinavian Journal of Statistics, 27(2):305–320, 2000. 9