jmlr jmlr2011 jmlr2011-37 jmlr2011-37-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jérémie Bigot, Rolando J. Biscay, Jean-Michel Loubes, Lillian Muñiz-Alvarez
Abstract: In this paper, we consider the Group Lasso estimator of the covariance matrix of a stochastic process corrupted by an additive noise. We propose to estimate the covariance matrix in a highdimensional setting under the assumption that the process has a sparse representation in a large dictionary of basis functions. Using a matrix regression model, we propose a new methodology for high-dimensional covariance matrix estimation based on empirical contrast regularization by a group Lasso penalty. Using such a penalty, the method selects a sparse set of basis functions in the dictionary used to approximate the process, leading to an approximation of the covariance matrix into a low dimensional space. Consistency of the estimator is studied in Frobenius and operator norms and an application to sparse PCA is proposed. Keywords: group Lasso, ℓ1 penalty, high-dimensional covariance estimation, basis expansion, sparsity, oracle inequality, sparse PCA
Anestis Antoniadis, Jeremie Bigot, and Theofanis Sapatinas. Wavelet estimators in nonparametric regression: A comparative simulation study. Journal of Statistical Software, 6(6):1–83, 6 2001. Francis R. Bach. Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res., 9:1179–1225, 2008. Peter J. Bickel and Elizaveta Levina. Regularized estimation of large covariance matrices. Ann. Statist., 36(1):199–227, 2008a. Peter J. Bickel and Elizaveta Levina. Covariance regularization by thresholding. Ann. Statist., 36 (6):2577–2604, 2008b. Peter J. Bickel, Ya’acov Ritov, and Alexandre B. Tsybakov. Simultaneous analysis of lasso and Dantzig selector. Ann. Statist., 37(4):1705–1732, 2009. J´ r´ mie Bigot, Rolando J. Biscay, Jean-Michel Loubes, and Lilian Mu˜ iz Alvarez. Nonparametric ee n estimation of covariance functions by model selection. Electronic Journal of Statistics, 4:822– 855, 2010. Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, 2004. ISBN 0-521-83378-7. Noel A. C. Cressie. Statistics for Spatial Data. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley & Sons Inc., New York, 1993. Alexandre d’Aspremont, Francis Bach, and Laurent El Ghaoui. Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res., 9:1269–1294, 2008. 3223 ˜ B IGOT, B ISCAY, L OUBES AND M U NIZ -A LVAREZ Kenneth R. Davidson and Stanislaw J. Szarek. Local operator theory, random matrices and Banach spaces. In Handbook of the Geometry of Banach Spaces, Vol. I, pages 317–366. North-Holland, Amsterdam, 2001. Chandler Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal., 7:1–46, 1970. Noureddine El Karoui. Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist., 36(6):2717–2756, 2008. Jianquin Fan, Yingying Fan, and Jinchi Lv. High dimensional covariance matrix estimation using a factor model. Journal of Econometrics, 147:186–197, 2008. Junzhou Huang and Tong Zhang. The benefit of group sparsity. Ann. Statist., 38(4):1978–2004, 2010. ISSN 0090-5364. Iain M. Johnstone. On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist., 29(2):295–327, 2001. Iain M. Johnstone and Arthur Y. Lu. On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 104(486):682–693, June 2009. Andre G. Journel. Kriging in terms of projections. J. Internat. Assoc. Mathematical Geol., 9(6): 563–586, 1977. Clifford Lam and Jianqing Fan. Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist., 37(6B):4254–4278, 2009. Elizaveta Levina, Adam Rothman, and Ji Zhu. Sparse estimation of large covariance matrices via a nested Lasso penalty. Ann. Appl. Stat., 2(1):245–263, 2008. Karim Lounici. Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat., 2:90–102, 2008. Karim Lounici, Massimiliano Pontil, Alexandre B. Tsybakov, and Sara van de Geer. Taking advantage of sparsity in multi-task learning. COLT, 2009. Karim Lounici, Massimiliano Pontil, Alexandre B. Tsybakov, and Sara van de Geer. Oracle Inequalities and Optimal Inference under Group Sparsity. Ann. Statist., to be published., 2011. Shahar Mendelson and Alain Pajor. Bernoulli, 12(5):761–773, 2006. On singular values of matrices with independent rows. Yuval Nardi and Alessandro Rinaldo. On the asymptotic properties of the group lasso estimator for linear models. Electron. J. Stat., 2:605–633, 2008. Adam J. Rothman, Peter J. Bickel, Elizaveta Levina, and Ji Zhu. Sparse permutation invariant covariance estimation. Electron. J. Stat., 2:494–515, 2008. 3224 G ROUP L ASSO E STIMATION OF H IGH - DIMENSIONAL C OVARIANCE M ATRICES Mark Schmidt, Kevin Murphy, Glenn Fung, and R´ mer Rosales. Structure learning in random fields o for heart motion abnormality detection (addendum). CVPR08, 2008. Michael L. Stein. Interpolation of Spatial Data. Some Theory for Kriging. Springer Series in Statistics. New York, NY: Springer. xvii, 247 p., 1999. Christopher K. Wikle and Noel Cressie. A dimension-reduced approach to space-time Kalman filtering. Biometrika, 86(4):815–829, 1999. ISSN 0006-3444. Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. J. Comput. Graph. Statist., 15(2):265–286, 2006. 3225