jmlr jmlr2013 jmlr2013-31 jmlr2013-31-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Kris De Brabanter, Jos De Brabanter, Bart De Moor, Irène Gijbels
Abstract: We present a fully automated framework to estimate derivatives nonparametrically without estimating the regression function. Derivative estimation plays an important role in the exploration of structures in curves (jump detection and discontinuities), comparison of regression curves, analysis of human growth data, etc. Hence, the study of estimating derivatives is equally important as regression estimation itself. Via empirical derivatives we approximate the qth order derivative and create a new data set which can be smoothed by any nonparametric regression estimator. We derive L1 and L2 rates and establish consistency of the estimator. The new data sets created by this technique are no longer independent and identically distributed (i.i.d.) random variables anymore. As a consequence, automated model selection criteria (data-driven procedures) break down. Therefore, we propose a simple factor method, based on bimodal kernels, to effectively deal with correlated data in the local polynomial regression framework. Keywords: nonparametric derivative estimation, model selection, empirical derivative, factor rule
J.L.O. Cabrera. locpol: Kernel Local Polynomial Regression, 2009. http://CRAN.R-project.org/package=locpol. R package version 0.4-0. URL R. Charnigo, M. Francoeur, M.P. Meng¨ c, A. Brock, M. Leichter, and C. Srinivasan. Derivatives of u¸ scattering profiles: tools for nanoparticle characterization. J. Opt. Soc. Am. A, 24(9):2578–2589, 2007. P. Chaudhuri and J.S. Marron. SiZer for exploration of structures in curves. J. Amer. Statist. Assoc., 94(447):807–823, 1999. C.K. Chu and J.S. Marron. Comparison of two bandwidth selectors with dependent errors. Ann. Statist., 19(4):1906–1918, 1991. K. De Brabanter, J. De Brabanter, J.A.K. Suykens, and B. De Moor. Kernel regression in the presence of correlated errors. J. Mach. Learn. Res., 12:1955–1976, 2011. M. Delecroix and A.C. Rosa. Nonparametric estimation of a regression function and its derivatives under an ergodic hypothesis. J. Nonparametr. Stat., 6(4):367–382, 2007. R.L. Eubank and P.L. Speckman. Confidence bands in nonparametric regression. J. Amer. Statist. Assoc., 88(424):1287–1301, 1993. J. Fan and I. Gijbels. Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation. J. R. Stat. Soc. Ser. B, 57(2):371–394, 1995. J. Fan and I. Gijbels. Local Polynomial Modeling and Its Applications. Chapman & Hall, 1996. T. Gasser and H.-G. M¨ ller. Estimating regression functions and their derivatives by the kernel u method. Scand. J. Statist., 11(3):171–185, 1984. I. Gijbels and A.-C. Goderniaux. Data-driven discontinuity detection in derivatives of a regression function. Communications in Statistics–Theory and Methods, 33:851–871, 2004. L. Gy¨ rfi, M. Kohler, A. Krzy˙ ak, and H. Walk. A Distribution-Free Theory of Nonparametric o z Regression. Springer, 2002. P. Hall, J.W. Kay, and D.M. Titterington. Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika, 77(3):521–528, 1990. W. H¨ rdle. Applied Nonparametric Regression. Cambridge University Press, 1990. a W. H¨ rdle and T. Gasser. On robust kernel estimation of derivatives of regression functions. Scand. a J. Statist., 12(3):233–240, 1985. A. Iserles. A First Course in the Numerical Analysis of Differential Equations. Cambridge University Press, 1996. R. Jarrow, D. Ruppert, and Y. Yu. Estimating the term structure of corporate debt with a semiparametric penalized spline model. J. Amer. Statist. Assoc., 99(465):57–66, 2004. 300 D ERIVATIVE E STIMATION WITH L OCAL P OLYNOMIAL F ITTING H.-G. M¨ ller. Nonparametric Regression Analysis of Longitudinal Data. Springer-Verlag, 1988. u H.-G. M¨ ller, U. Stadtm¨ ller, and T. Schmitt. Bandwidth choice and confidence intervals for derivau u tives of noisy data. Biometrika, 74(4):743–749, 1987. J. Newell and J. Einbeck. A comparative study of nonparametric derivative estimators. Proc. of the 22nd International Workshop on Statistical Modelling, 2007. J. Opsomer, Y. Wang, and Y. Yang. Nonparametric regression with correlated errors. Statist. Sci., 16(2):134–153, 2001. C. Park and K.-H. Kang. SiZer analysis for the comparison of regression curves. Comput. Statist. Data Anal., 52(8):3954–3970, 2008. K. Patan. Artificial Neural Networks for the Modelling and Fault Diagnosis of Technical Processes. Springer-Verlag, 2008. J. Ramsay. Derivative estimation. StatLib – S-News, Thursday, March 12, 1998: http://www.math.yorku.ca/Who/Faculty/Monette/S-news/0556.html, 1998. J.O. Ramsay and B.W. Silverman. Applied Functional Data Analysis. Springer-Verlag, 2002. J. Ramsey and B. Ripley. pspline: Penalized Smoothing Splines, 2010. http://CRAN.R-project.org/package=pspline. R package version 1.0-14. URL V. Rondonotti, J.S. Marron, and C. Park. SiZer for time series: A new approach to the analysis of trends. Electron. J. Stat., 1:268–289, 2007. D. Ruppert and M.P. Wand. Multivariate locally weighted least squares regression. Ann. Statist., 22 (3):1346–1370, 1994. J.S. Simonoff. Smoothing Methods in Statistics. Springer-Verlag, 1996. C. Stone. Additive regression and other nonparametric models. Ann. Statist., 13(2):689–705, 1985. A.B. Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009. G. Wahba and Y. Wang. When is the optimal regularization parameter insensitive to the choice of loss function? Comm. Statist. Theory Methods, 19(5):1685–1700, 1990. M.P. Wand and M.C. Jones. Kernel Smoothing. Chapman & Hall, 1995. S. Zhou and D.A. Wolfe. On derivative estimation in spline regression. Statist. Sinica, 10(1): 93–108, 2000. 301