nips nips2010 nips2010-154 nips2010-154-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Alessandro Chiuso, Gianluigi Pillonetto
Abstract: We introduce a new Bayesian nonparametric approach to identification of sparse dynamic linear systems. The impulse responses are modeled as Gaussian processes whose autocovariances encode the BIBO stability constraint, as defined by the recently introduced “Stable Spline kernel”. Sparse solutions are obtained by placing exponential hyperpriors on the scale factors of such kernels. Numerical experiments regarding estimation of ARMAX models show that this technique provides a definite advantage over a group LAR algorithm and state-of-the-art parametric identification techniques based on prediction error minimization. 1
[1] L. Ljung. System Identification - Theory For the User. Prentice Hall, 1999.
[2] J. Mohammadpour and K.M. Grigoriadis. Springer, 2010. Efficient Modeling and Control of Large-scale Systems.
[3] T. J. Hastie and R. J. Tibshirani. Generalized additive models. In Monographs on Statistics and Applied Probability, volume 43. Chapman and Hall, London, UK, 1990.
[4] D. Donoho. Compressed sensing. IEEE Trans. on Information Theory, 52(4):1289–1306, 2006.
[5] H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19:716–723, 1974.
[6] G. Schwarz. Estimating the dimension of a model. The Annals of Statistics, 6:461–464, 1978.
[7] R. Tibshirani. Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society, Series B., 58, 1996.
[8] B. Efron, T. Hastie, L. Johnstone, and R. Tibshirani. Least angle regression. Annals of Statistics, 32:407– 499, 2004.
[9] P. Zhao and B. Yu. On model selection consistency of lasso. Journal of Machine Learning Research, 7:2541–2563, 2006.
[10] H. Zou. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101:1418–1429, 2006.
[11] Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68:49–67, 2006.
[12] F.R. Bach. Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res., 9:1179– 1225, 2008.
[13] C. A. Micchelli and M. Pontil. Learning the kernel function via regularization. Journal of Machine Learning Research, 6:1099–1125, 2005.
[14] H. Wang, G. Li, and C.L. Tsai. Regression coefficient and autoregressive order shrinkage and selection via the lasso. Journal Of The Royal Statistical Society Series B, 69(1):63–78, 2007.
[15] Nan-Jung Hsu, Hung-Lin Hung, and Ya-Mei Chang. Subset selection for vector autoregressive processes using lasso. Computational Statistics and Data Analysis, 52:3645–3657, 2008.
[16] G. Pillonetto and G. De Nicolao. A new kernel-based approach for linear system identification. Automatica, 46(1):81–93, 2010.
[17] G. Pillonetto, A. Chiuso, and G. De Nicolao. Prediction error identification of linear systems: a nonparametric Gaussian regression approach. Automatica (in press), 2011.
[18] C.E. Rasmussen and C.K.I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
[19] N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337–404, 1950.
[20] G. Wahba. Support vector machines, reproducing kernel Hilbert spaces and randomized GACV. Technical Report 984, Department of Statistics, University of Wisconsin, 1998.
[21] G. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications, 33(1):82–95, 1971.
[22] A. J. Smola and B. Sch¨ lkopf. Bayesian kernel methods. In S. Mendelson and A. J. Smola, editors, o Machine Learning, Proceedings of the Summer School, Australian National University, pages 65–117, Berlin, Germany, 2003. Springer-Verlag.
[23] G. Wahba. Spline models for observational data. SIAM, Philadelphia, 1990.
[24] G.C. Goodwin, M. Gevers, and B. Ninness. Quantifying the error in estimated transfer functions with application to model order selection. IEEE Transactions on Automatic Control, 37(7):913–928, 1992.
[25] F. Dinuzzo. Kernel machines with two layers and multiple kernel learning. Technical report, Preprint arXiv:1001.2709, 2010. Available at http://www-dimat.unipv.it/ dinuzzo.
[26] L. Ljung. System Identification Toolbox V7.1 for Matlab. Natick, MA: The MathWorks, Inc., 2007. 9