jmlr jmlr2012 jmlr2012-117 jmlr2012-117-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lan Xue, Annie Qu
Abstract: The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. It is important to identify significant covariates associated with response variables, especially for high-dimensional settings where the number of covariates can be larger than the sample size. We consider model selection in the high-dimensional setting and adopt difference convex programming to approximate the L0 penalty, and we investigate the global optimality properties of the varying-coefficient estimator. The challenge of the variable selection problem here is that the dimension of the nonparametric form for the varying-coefficient modeling could be infinite, in addition to dealing with the high-dimensional linear covariates. We show that the proposed varying-coefficient estimator is consistent, enjoys the oracle property and achieves an optimal convergence rate for the non-zero nonparametric components for high-dimensional data. Our simulations and numerical examples indicate that the difference convex algorithm is efficient using the coordinate decent algorithm, and is able to select the true model at a higher frequency than the least absolute shrinkage and selection operator (LASSO), the adaptive LASSO and the smoothly clipped absolute deviation (SCAD) approaches. Keywords: coordinate decent algorithm, difference convex programming, L0 - regularization, large-p small-n, model selection, nonparametric function, oracle property, truncated L1 penalty
L. An and P. Tao. Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. Journal of Global Optimization, 11:253-285, 1997. L. Breiman and A. Cutler. A deterministic algorithm for global optimization. Mathematical Programming, 58:179-199, 1993. C. de Boor. A Practical Guide to Splines. Springer, New York, 2001. J. Fan and T. Huang. Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli, 11:1031-1057, 2005. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96:1348-1360, 2001. J. Fan and H. Peng. Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics, 32:928-961, 2004. J. Fan and J. Zhang. Two-step estimation of functional linear models with applications to longitudinal data. Journal of the Royal Statistical Society, Series B, 62:303-322, 2000. T. Hastie and R. Tibshirani. Varying-coefficient models. Journal of the Royal Statistical Society, Series B, 55:757-796, 1993. D. R. Hunter and R. Li. Variable selection using MM algorithms. Annals of Statistics, 33:16171642, 2005. D. R. Hoover, J. A. Rice, C. O. Wu, and L. Yang. Nonparametric smoothing estimates of timevarying coefficient models with longitudinal data. Biometrika, 85:809-822, 1998. J. Huang, J. L. Horowitz, and F. Wei. Variable selection in nonparametric additive models. Annals of Statistics, 38:2282-2313, 2010. J. Z. Huang, C. O. Wu, and L. Zhou. Varying-coefficient models and basis function approximations for the analysis of repeated measurements. Biometrika, 89:111-128, 2002. J. Z. Huang, C. O. Wu, and L. Zhou. Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statistica Sinica, 14:763-788, 2004. Y. Kim, H. Choi, and H. Oh. Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103:1665-1673, 2008. Y. Liu, X. Shen, and W. Wong. Computational development of psi-learning. Proc SIAM 2005 Int. Data Mining Conf., 1-12, 2005. 1997 X UE AND Q U A. Qu, and R. Li. Quadratic inference functions for varying-coefficient models with longitudinal data. Biometrics, 62:379-391, 2006. J. O. Ramsay, and B. W. Silverman. Functional Data Analysis. Springer-Verlag: New York, 1997. X. Shen, W. Pan, Y. Zhu. Likelihood-based selection and sharp parameter estimation. Journal of the American Statistical Association, 107:223-232, 2012. X. Shen, G. C. Tseng, X. Zhang, and W. H. Wong. On ψ-learning. Journal of the American Statistical Association, 98:724-734, 2003. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1996. S. Van de Geer. High-dimensional generalized linear models and the Lasso. Annals of Statistics, 36:614-645, 2008. H. Wang, and Y. Xia. Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association, 104:747-757, 2009. L. Wang, H. Li, and J. Z. Huang. Variable selection in nonparametric varying-coefficient models for analysis of repeated measurements. Journal of the American Statistical Association, 103:15561569, 2008. F. Wei, J. Huang, and H. Li. Variable selection and estimation in high-dimensional varying coefficient models. Statistica Sinica, 21:1515-1540, 2011. C. O. Wu, and C. Chiang. Kernel smoothing on varying coefficient models with longitudinal dependent variable. Statistica Sinica, 10:433-456, 2000. L. Xue, A. Qu, and J. Zhou. Consistent model selection for marginal generalized additive model for correlated data. Journal of the American Statistical Association, 105:1518-1530, 2010. M. Yuan, and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68:49-67, 2006. C. H. Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38:894-942, 2010. P. Zhao, and B. Yu. On model selection consistency of Lasso. Journal of Machine Learning Research, 7:2541-2563, 2006. S. Zhou, X. Shen, and D. A. Wolfe. Local asymptotics for regression splines and confidence regions. Annals of Statistics, 26:1760-1782, 1998. H. Zou, and R. Li. One-step sparse estimates in nonconcave penalized likelihood models. Annals of Statistics, 36:1509-1533, 2008. 1998