nips nips2008 nips2008-174 nips2008-174-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Stéphan J. Clémençcon, Nicolas Vayatis
Abstract: ROC curves are one of the most widely used displays to evaluate performance of scoring functions. In the paper, we propose a statistical method for directly optimizing the ROC curve. The target is known to be the regression function up to an increasing transformation and this boils down to recovering the level sets of the latter. We propose to use classifiers obtained by empirical risk minimization of a weighted classification error and then to construct a scoring rule by overlaying these classifiers. We show the consistency and rate of convergence to the optimal ROC curve of this procedure in terms of supremum norm and also, as a byproduct of the analysis, we derive an empirical estimate of the optimal ROC curve. 1
[AA07] J.-Y. Audibert and A.Tsybakov. Fast learning rates for plug-in classifiers. Annals of statistics, 35(2):608–633, 2007. [AGH+ 05] S. Agarwal, T. Graepel, R. Herbrich, S. Har-Peled, and D. Roth. Generalization bounds for the area under the ROC curve. J. Mach. Learn. Res., 6:393–425, 2005. [BBL05] S. Boucheron, O. Bousquet, and G. Lugosi. Theory of Classification: A Survey of Some Recent Advances. ESAIM: Probability and Statistics, 9:323–375, 2005. [BCT07] M. Barreno, A.A. Cardenas, and J.D. Tygar. Optimal ROC curve for a combination of classifiers. In NIPS’07, 2007. [BDH06] F.R. Bach, D.Heckerman, and Eric Horvitz. Considering cost asymmetry in learning classifiers. Journal of Machine Learning Research, 7:1713–1741, 2006. [Cav97] L. Cavalier. Nonparametric estimation of regression level sets. Statistics, 29:131–160, 1997. [CLV08] S. Cl´ mencon, G. Lugosi, and N. Vayatis. Ranking and empirical risk minimization of e ¸ U-statistics. The Annals of Statistics, 36(2):844–874, 2008. [CV07] S. Cl´ mencon and N. Vayatis. Ranking the best instances. Journal of Machine Learning e ¸ Research, 8:2671–2699, 2007. [CV08] S. Cl´ mencon and N. Vayatis. Tree-structured ranking rules and approximation of the e ¸ optimal ROC curve. Technical Report hal-00268068, HAL, 2008. [dB01] C. de Boor. A practical guide to splines. Springer, 2001. [Ega75] J.P. Egan. Signal Detection Theory and ROC Analysis. Academic Press, 1975. [FISS03] Y. Freund, R. D. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933–969, 2003. [RV06] P. Rigollet and R. Vert. Fast rates for plug-in estimators of density level sets. Technical Report arXiv:math/0611473v2, arXiv:math/0611473v2, 2006. [Tsy04] A. Tsybakov. Optimal aggregation of classifiers in statistical learning. Annals of Statistics, 32(1):135–166, 2004. [vT68] H.L. van Trees. Detection, Estimation, and Modulation Theory, Part I. Wiley, 1968. [WN07] R. Willett and R. Nowak. Minimax optimal level set estimation. IEEE Transactions on Image Processing, 16(12):2965–2979, 2007. 8