jmlr jmlr2009 jmlr2009-5 jmlr2009-5-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Gilles Blanchard, Étienne Roquain
Abstract: In the context of multiple hypothesis testing, the proportion π0 of true null hypotheses in the pool of hypotheses to test often plays a crucial role, although it is generally unknown a priori. A testing procedure using an implicit or explicit estimate of this quantity in order to improve its efficency is called adaptive. In this paper, we focus on the issue of false discovery rate (FDR) control and we present new adaptive multiple testing procedures with control of the FDR. In a first part, assuming independence of the p-values, we present two new procedures and give a unified review of other existing adaptive procedures that have provably controlled FDR. We report extensive simulation results comparing these procedures and testing their robustness when the independence assumption is violated. The new proposed procedures appear competitive with existing ones. The overall best, though, is reported to be Storey’s estimator, albeit for a specific parameter setting that does not appear to have been considered before. In a second part, we propose adaptive versions of step-up procedures that have provably controlled FDR under positive dependence and unspecified dependence of the p-values, respectively. In the latter case, while simulations only show an improvement over non-adaptive procedures in limited situations, these are to our knowledge among the first theoretically founded adaptive multiple testing procedures that control the FDR when the p-values are not independent. Keywords: multiple testing, false discovery rate, adaptive procedure, positive regression dependence, p-values
Y. Benjamini and R. Heller. False discovery rates for spatial signals. Journal of the American Statistical Association, 102(480):1272–1281, 2007. Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Ser. B, 57(1):289–300, 1995. Y. Benjamini and Y. Hochberg. On the adaptive control of the false discovery rate in multiple testing with independent statistics. Journal of Educational and Behavioral Statistics, 25:60–83, 2000. Y. Benjamini and D. Yekutieli. The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29(4):1165–1188, 2001. 2869 B LANCHARD AND ROQUAIN Y. Benjamini, A. M. Krieger, and D. Yekutieli. Adaptive linear step-up procedures that control the false discovery rate. Biometrika, 93(3):491–507, 2006. M. A. Black. A note on the adaptive control of false discovery rates. Journal of the Royal Statistical Society, Series B, 66(2):297–304, 2004. G. Blanchard and F. Fleuret. Occam’s hammer. In N.H. Bshouty and C. Gentile, editors, Proceedings of the 20th. Conference on Learning Theory (COLT 2007), volume 4539 of Springer Lecture Notes on Computer Science, pages 112–126, 2007. G. Blanchard and E. Roquain. Two simple sufficient conditions for FDR control. Electronic Journal of Statistics, 2:963–992, 2008. S. Dudoit, J. Shaffer, and J. Boldrick. Multiple hypothesis testing in microarray experiments. Statistical Science, 18(1):71–103, 2003. B. Efron, R. Tibshirani, J. Storey, and V. Tusher. Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association, 96(456):1151–1160, 2001. A. Farcomeni. Some results on the control of the false discovery rate under dependence. Scandinavian Journal of Statistics, 34(2):275–297, 2007. H. Finner and M. Roters. On the false discovery rate and expected type I errors. Biometrical Journal, 43(8):985–1005, 2001. H. Finner, T. Dickhaus, and M. Roters. Dependency and false discovery rate: Asymptotics. Annals of Statistics, 35(4):1432–1455, 2007. H. Finner, T. Dickhaus, and M. Roters. On the false discovery rate and an asymptotically optimal rejection curve. Annals of Statistics, 37(2):596–618, 2009. Y. Gavrilov, Y. Benjamini, and S. K. Sarkar. An adaptive step-down procedure with proven FDR control under independence. Annals of Statistics, 37(2):619–629, 2009. C. Genovese and L. Wasserman. Operating characteristics and extensions of the false discovery rate procedure. Journal of the Royal Statistical Society, Series B, 64(3):499–517, 2002. C. Genovese and L. Wasserman. A stochastic process approach to false discovery control. Annals of Statistics, 32(3):1035–1061, 2004. C. Genovese, K. Roeder, and L. Wasserman. False discovery control with p-value weighting. Biometrika, 93(3):509–524, 2006. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13–30, 1963. S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):65–70, 1979. J. Jin. Proportion of non-zero normal means: universal oracle equivalences and uniformly consistent estimators. Journal of the Royal Statistical Society, Series B, 70(3):461–493, 2008. 2870 A DAPTIVE FDR C ONTROL UNDER I NDEPENDENCE AND D EPENDENCE J. Jin and T. Cai. Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons. Journal of the American Statistical Association, 102(478):495–506, 2007. E. L. Lehmann. Some concepts of dependence. Annals of Mathematical Statistics, 37:1137–1153, 1966. N. Meinshausen and J. Rice. Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. Annals of Statistics, 34(1):373–393, 2006. P. Neuvial. Asymptotic properties of false discovery rate controlling procedures under independence. Electronic Journal of Statistics, 2:1065–1110, 2008. E. Roquain. Exceptional Motifs in Heterogeneous Sequences. Contributions to Theory and Methodology of Multiple Testing. PhD thesis, Universit´ Paris XI, 2007. e E. Roquain and M. van de Wiel. Optimal weighting for false discovery rate control. Electronic Journal of Statistics, 3:678–711, 2009. S. K. Sarkar. Two-stage stepup procedures controlling FDR. Journal of Statistical Planning and Inference, 138(4):1072–1084, 2008a. S. K. Sarkar. On methods controlling the false discovery rate. Sankhya, Series A, 70:135–168, 2008b. C. Scott and G. Blanchard. Novelty detection: unlabaled data definitely help. In D. van Dyk and M. Welling, editors, Proceedings of the 12th. International Conference on Artificial Intelligence and Statistics (AISTATS 2009), volume 5 of JMLR Workshop and Conference Proceedings, pages 464–471, 2009. E. Spjøtvoll. On the optimality of some multiple comparison procedures. Annals of Mathematical Statistics, 43(2):398–411, 1972. J. Storey. A direct approach to false discovery rates. Journal of the Royal Statistical Society, Series B, 64(3):479–498, 2002. J. Storey. The optimal discovery procedure: a new approach to simultaneous significance testing. Journal of the Royal Statistical Society, Series B, 69(3):347–368, 2007. J. Storey and R. Tibshirani. SAM thresholding and false discovery rates for detecting differential gene expression in DNA microarrays. In The Analysis of Gene Expression Data, Statistics for Biology and Health, pages 272–290. Springer, New York, 2003. J. Storey, J. Taylor, and D. Siegmund. Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach. Journal of the Royal Statistical Society, Series B, 66(1):187–205, 2004. W. Sun and T. Cai. Oracle and adaptive compound decision rules for false discovery rate control. Journal of the American Statistical Association, 102(479):901–912, 2007. L. Wasserman and K. Roeder. Weighted hypothesis testing. Technical report 830, Dept. of statistics, Carnegie Mellon University, 2006. ArXiv preprint math/0604172v1. URL: http://arxiv.org/abs/math.ST/0604172. 2871