jmlr jmlr2011 jmlr2011-97 jmlr2011-97-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Mladen Kolar, John Lafferty, Larry Wasserman
Abstract: We sharply characterize the performance of different penalization schemes for the problem of selecting the relevant variables in the multi-task setting. Previous work focuses on the regression problem where conditions on the design matrix complicate the analysis. A clearer and simpler picture emerges by studying the Normal means model. This model, often used in the field of statistics, is a simplified model that provides a laboratory for studying complex procedures. Keywords: high-dimensional inference, multi-task learning, sparsity, normal means, minimax estimation
´ ´e David Aldous. Exchangeability and related topics. In Ecole d’Et´ de Probabilit´ s de Saint-Flour e XIII 1983, pages 1–198. Springer, Berlin, 1985. Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243–272, 2008. 2433 KOLAR , L AFFERTY AND WASSERMAN Sylvain Arlot and Francis Bach. Data-driven calibration of linear estimators with minimal penalties. In Advances in Neural Information Processing Systems 22, pages 46–54. 2009. Yannick Baraud. Non-asymptotic minimax rates of testing in signal detection. Bernoulli, 8(5): 577–606, 2002. Larry Brown and Mark Low. Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist., 24(6):2384–2398, 1996. Jianqing Fan and Runze Li. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Ass., 96:1348–1360, 2001. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. A note on the group lasso and a sparse group lasso. Technical report, Stanford, 2010. Available at arXiv:1001.0736. Seyoung Kim, Kyung-Ah Sohn, and Eric P. Xing. A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics, 25(12):i204–212, 2009. Mladen Kolar and Eric P. Xing. Ultra-high dimensional multiple output learning with simultaneous orthogonal matching pursuit: Screening approach. In AISTATS ’10: Proc. 13th Int’l Conf. on Artifical Intelligence and Statistics, pages 413–420, 2010. Han Liu, Mark Palatucci, and Jian Zhang. Blockwise coordinate descent procedures for the multitask lasso, with applications to neural semantic basis discovery. In ICML ’09: Proc. 26th Int’l Conf. on Machine Learning, pages 649–656, New York, NY, USA, 2009. Karim Lounici, Massimiliano Pontil, Alexandre Tsybakov, and Sara van de Geer. Taking advantage of sparsity in Multi-Task learning. In COLT ’09: Proc. Conf. on Learning Theory, 2009. Karim Lounici, Massimiliano Pontil, Alexandre B Tsybakov, and Sara van de Geer. Oracle inequalities and optimal inference under group sparsity. Preprint, 2010. Available at arXiv:1007.1771. Sahand Negahban and Martin Wainwright. Phase transitions for high-dimensional joint support recovery. In Advances in Neural Information Processing Systems 21, pages 1161–1168, 2009. Michael Nussbaum. Asymptotic equivalence of density estimation and gaussian white noise. Ann. Statist., 24(6):2399–2430, 1996. Guillaume Obozinski, Martin Wainwright, and Michael Jordan. Support union recovery in highdimensional multivariate regression. Ann. Statist., 39(1):1–47, 2011. Alexandre Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009. Berwin Turlach, William Venables, and Stephen Wright. Simultaneous variable selection. Technometrics, 47(3):349–363, 2005. ISSN 0040-1706. Sara van de Geer and Peter B¨ hlmann. On the conditions used to prove oracle results for the lasso. u Elec. J. Statist., 3:1360–1392, 2009. Jian Zhang. A Probabilistic Framework for Multitask Learning. PhD thesis, Carnegie Mellon University, 2006. 2434 U NION S UPPORT R ECOVERY Peng Zhao and Bin Yu. On model selection consistency of lasso. J. Mach. Learn. Res., 7:2541– 2563, 2006. ISSN 1533-7928. Hui Zou and Ming Yuan. The F∞ -norm support vector machine. Stat. Sin, 18:379–398, 2008. 2435