nips nips2001 nips2001-155 nips2001-155-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Peter Meinicke, Helge Ritter
Abstract: We suggest a nonparametric framework for unsupervised learning of projection models in terms of density estimation on quantized sample spaces. The objective is not to optimally reconstruct the data but instead the quantizer is chosen to optimally reconstruct the density of the data. For the resulting quantizing density estimator (QDE) we present a general method for parameter estimation and model selection. We show how projection sets which correspond to traditional unsupervised methods like vector quantization or PCA appear in the new framework. For a principal component quantizer we present results on synthetic and realworld data, which show that the QDE can improve the generalization of the kernel density estimator although its estimate is based on significantly lower-dimensional projection indices of the data.
[1] J. M. Buhmann and N. Tishby. Empirical risk approximation: A statistical learning theory of data clustering. In C. M. Bishop, editor, Neural Networks and Machine Learning, pages 57–68. Springer, Berlin Heidelberg New York, 1998.
[2] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B, 39:1–38, 1977.
[3] Stuart Geman and Chii-Ruey Hwang. Nonparametric maximum likelihood estimation by the method of sieves. The Annals of Statistics, 10(2):401–414, 1982.
[4] T. Hastie and W. Stuetzle. Principal curves. Journal of the American Statistical Association, 84:502–516, 1989.
[5] M. C. Jones, J. S. Marron, and S. J. Sheather. A brief survey of bandwidth selection for density estimation. Journal of the American Statistical Association, 91(433):401– 407, 1996.
[6] B. K´ gl, A. Krzyzak, T. Linder, and K. Zeger. Learning and design of principal e curves. IEEE Transaction on Pattern Analysis and Machine Intelligence, 22(3):281– 297, 2000.
[7] Peter Meinicke. Unsupervised Learning in a Generalized Regression Framework. PhD thesis, Universitaet Bielefeld, 2000. http://archiv.ub.unibielefeld.de/disshabi/2000/0033/.
[8] Peter Meinicke and Helge Ritter. Resolution-based complexity control for Gaussian mixture models. Neural Computation, 13(2):453–475, 2001.
[9] K. Rose, E. Gurewitz, and G. C. Fox. Statistical mechanics and phase transitions in clustering. Physical Review Letters, 65(8):945–948, 1990.
[10] Sam Roweis and Zoubin Ghahramani. A unifying review of linear Gaussian models. Neural Computation, 11(2):305–345, 1999.
[11] D. W. Scott. Multivariate Density Estimation. Wiley, 1992.
[12] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman and Hall, London and New York, 1986.
[13] Alex J. Smola, Robert C. Williamson, Sebastian Mika, and Bernhard Sch¨ lkopf. Rego ularized principal manifolds. In Proc. 4th European Conference on Computational Learning Theory, volume 1572, pages 214–229. Springer-Verlag, 1999.
[14] Vladimir N. Vapnik and Sayan Mukherjee. Support vector method for multivariate density estimation. In S. A. Solla, T. K. Leen, and K.-R. M¨ ller, editors, Advances in u Neural Information Processing Systems, volume 12, pages 659–665. The MIT Press, 2000.