nips nips2011 nips2011-240 nips2011-240-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Daniel Hernández-lobato, Jose M. Hernández-lobato, Pierre Dupont
Abstract: Multi-class Gaussian Process Classifiers (MGPCs) are often affected by overfitting problems when labeling errors occur far from the decision boundaries. To prevent this, we investigate a robust MGPC (RMGPC) which considers labeling errors independently of their distance to the decision boundaries. Expectation propagation is used for approximate inference. Experiments with several datasets in which noise is injected in the labels illustrate the benefits of RMGPC. This method performs better than other Gaussian process alternatives based on considering latent Gaussian noise or heavy-tailed processes. When no noise is injected in the labels, RMGPC still performs equal or better than the other methods. Finally, we show how RMGPC can be used for successfully identifying data instances which are difficult to classify correctly in practice. 1
[1] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2006.
[2] Christopher K. I. Williams and David Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1342–1351, 1998.
[3] Hyun-Chul Kim and Zoubin Ghahramani. Bayesian Gaussian process classification with the EM-EP algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):1948–1959, 2006.
[4] R.M Neal. Regression and classification using Gaussian process priors. Bayesian Statistics, 6:475–501, 1999.
[5] Matthias Seeger and Michael I. Jordan. Sparse Gaussian process classification with multiple classes. Technical report, University of California, Berkeley, 2004.
[6] M. Opper and O. Winther. Gaussian process classification and SVM: Mean field results. In P. Bartlett, B.Schoelkopf, D. Schuurmans, and A. Smola, editors, Advances in large margin classifiers, pages 43–65. MIT Press, 2000.
[7] Daniel Hern´ ndez-Lobato and Jos´ Miguel Hern´ ndez-Lobato. Bayes machines for binary a e a classification. Pattern Recognition Letters, 29(10):1466–1473, 2008.
[8] Hyun-Chul Kim and Zoubin Ghahramani. Outlier robust Gaussian process classification. In Structural, Syntactic, and Statistical Pattern Recognition, volume 5342 of Lecture Notes in Computer Science, pages 896–905. Springer Berlin / Heidelberg, 2008.
[9] Fabian L. Wauthier and Michael I. Jordan. Heavy-Tailed Process Priors for Selective Shrinkage. In J. Lafferty, C. K. I. Williams, R. Zemel, J. Shawe-Taylor, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 2406–2414. 2010.
[10] Thomas Minka. A Family of Algorithms for approximate Bayesian Inference. PhD thesis, Massachusetts Institute of Technology, 2001.
[11] A. Asuncion and D.J. Newman. UCI machine learning repository, 2007.
[12] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines, 2001.
[13] Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, August 2006.
[14] T. Minka and J. Lafferty. Expectation-propagation for the generative aspect model. In Adnan Darwiche and Nir Friedman, editors, Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, pages 352–359. Morgan Kaufmann, 2002.
[15] Malte Kuss and Carl Edward Rasmussen. Assessing approximate inference for binary Gaussian process classification. Journal of Machine Learning Research, 6:1679–1704, 2005.
[16] H Nickisch and CE Rasmussen. Approximations for binary Gaussian process classification. Journal of Machine Learning Research, 9:2035–2078, 10 2008.
[17] Marcel Van Gerven, Botond Cseke, Robert Oostenveld, and Tom Heskes. Bayesian source localization with the multivariate Laplace prior. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1901–1909, 2009.
[18] Matthias Seeger. Expectation propagation for exponential families. Technical report, Department of EECS, University of California, Berkeley, 2006. 9