jmlr jmlr2008 jmlr2008-7 jmlr2008-7-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Glenn Shafer, Vladimir Vovk
Abstract: Conformal prediction uses past experience to determine precise levels of conÄ?Ĺš dence in new predictions. Given an error probability ĂŽÄž, together with a method that makes a prediction y of a label Ă‹† y, it produces a set of labels, typically containing y, that also contains y with probability 1 − ĂŽÄž. Ă‹† Conformal prediction can be applied to any method for producing y: a nearest-neighbor method, a Ă‹† support-vector machine, ridge regression, etc. Conformal prediction is designed for an on-line setting in which labels are predicted successively, each one being revealed before the next is predicted. The most novel and valuable feature of conformal prediction is that if the successive examples are sampled independently from the same distribution, then the successive predictions will be right 1 − ĂŽÄž of the time, even though they are based on an accumulating data set rather than on independent data sets. In addition to the model under which successive examples are sampled independently, other on-line compression models can also use conformal prediction. The widely used Gaussian linear model is one of these. This tutorial presents a self-contained account of the theory of conformal prediction and works through several numerical examples. A more comprehensive treatment of the topic is provided in Algorithmic Learning in a Random World, by Vladimir Vovk, Alex Gammerman, and Glenn Shafer (Springer, 2005). Keywords: conÄ?Ĺš dence, on-line compression modeling, on-line learning, prediction regions
Eugene A. Asarin. Some properties of Kolmogorov δ-random Ä?Ĺš nite sequences. Theory of Probability and its Applications, 32:507–508, 1987. Eugene A. Asarin. On some properties of Ä?Ĺš nite objects random in the algorithmic sense. Soviet Mathematics Doklady, 36:109–112, 1988. Peter J. Bickel and Kjell A. Doksum. Mathematical Statistics: Basic Ideas and Selected Topics. Volume I. Prentice Hall, Upper Saddle River, New Jersey, second edition, 2001. Robert G. Cowell, A. Philip Dawid, Steffen L. Lauritzen, and David J. Spiegelhalter. Probabilistic Networks and Expert Systems. Springer, New York, 1999. Emanuel Czuber. Wahrscheinlichkeitsrechnung und ihre Anwendung auf Fehlerausgleichung, Statistik, und Lebensversicherung. Volume I. Wahrscheinlichkeitstheorie, Fehlerausgleichung, Kollektivmasslehre. Teubner, Leipzig and Berlin, third edition, 1914. Arthur P. Dempster. Elements of Continuous Multivariate Analysis. Addison-Wesley, Reading, Massachusetts, 1969. Norman R. Draper and Harry Smith. Applied Regression Analysis. Wiley, New York, third edition, 1998. Bradley Efron. Student’s t-test under symmetry conditions. Journal of the American Statistical Association, 64:1278–1302, 1969. Ronald A. Fisher. Applications of Student’s distribution. Metron, 5:90–104, 1925. Ronald A. Fisher. The Ä?Ĺš ducial argument in statistical inference. Annals of Eugenics, 6:391–398, 1935. Ronald A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179–188, 1936. Ronald A. Fisher. Statistical Methods and ScientiÄ?Ĺš c Inference. Hafner, New York, third edition, 1973. First edition, 1956. Alex Gammerman and Vladimir Vovk. Hedging predictions in machine learning. Computer Journal, 50:151–163, 2007. 420 T UTORIAL ON C ONFORMAL P REDICTION Edwin Hewitt and Leonard J. Savage. Symmetric measures on Cartesian products. Transactions of the American Mathematical Society, 80:470–501, 1955. Steffen L. Lauritzen. Extremal Families and Systems of SufÄ?Ĺš cient Statistics. Volume 49 of Lecture Notes in Statistics. Springer, New York, 1988. Erich L. Lehmann. Testing Statistical Hypotheses. Wiley, New York, second edition, 1986. First edition, 1959. Per Martin-L¨ f. Repetitive structures and the relation between canonical and microcanonical diso tributions in statistics and statistical mechanics. In Ole E. Barndorff-Nielsen, Preben BlÄ‚Ĺšsild, and Geert Schou, editors, Proceedings of Conference on Foundational Questions in Statistical Inference, pages 271–294, Aarhus, 1974. Memoirs 1. Jerzy Neyman. Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society, Series A, 237:333–380, 1937. Geoff K. Robinson. Some counterexamples to the theory of conÄ?Ĺš dence intervals. Biometrika, 62: 155–161, 1975. Thomas P. Ryan. Modern Regression Methods. Wiley, New York, 1997. George A. F. Seber and Alan J. Lee. Linear Regression Analysis. Wiley, Hoboken, NJ, second edition, 2003. Glenn Shafer. From Cournot’s principle to market efÄ?Ĺš ciency. In Jean-Philippe Touffut, editor, Augustin Cournot, Economic Models and Rationality. Edward Elgar, Cheltenham, 2007. Glenn Shafer and Vladimir Vovk. Probability and Finance: It’s only a Game! Wiley, New York, 2001. Dylan S. Small, Joseph L. Gastwirth, Abba M. Krieger, and Paul R. Rosenbaum. R-estimates vs. GMM: A theoretical case study of validity and efÄ?Ĺš ciency. Statistical Science, 21:363–375, 2006. Student (William S. Gossett). On the probable error of a mean. Biometrika, 6:1–25, 1908. John W. Tukey. Sunset salvo. American Statistician, 40:72–76, 1986. Vladimir N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998. Vladimir Vovk, Alex Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World. Springer, New York, 2005. 421