nips nips2010 nips2010-250 nips2010-250-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ernesto D. Vito, Lorenzo Rosasco, Alessandro Toigo
Abstract: In this paper we consider the problem of learning from data the support of a probability distribution when the distribution does not have a density (with respect to some reference measure). We propose a new class of regularized spectral estimators based on a new notion of reproducing kernel Hilbert space, which we call “completely regular”. Completely regular kernels allow to capture the relevant geometric and topological properties of an arbitrary probability space. In particular, they are the key ingredient to prove the universal consistency of the spectral estimators and in this respect they are the analogue of universal kernels for supervised problems. Numerical experiments show that spectral estimators compare favorably to state of the art machine learning algorithms for density support estimation.
[1] P. M. Anselone. Collectively compact operator approximation theory and applications to integral equations. Prentice-Hall Inc., Englewood Cliffs, N. J., 1971.
[2] N. Aronszajn. Theory of reproducing kernels. Trans. Amer. Math. Soc., 68:337–404, 1950.
[3] M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res., 7:2399–2434, 2006.
[4] C. Berg, J. Christensen, and P. Ressel. Harmonic analysis on semigroups, volume 100 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1984.
[5] G. Biau, B. Cadre, D. Mason, and Bruno Pelletier. Asymptotic normality in density support estimation. Electron. J. Probab., 14:no. 91, 2617–2635, 2009.
[6] S. Canu, Y. Grandvalet, V. Guigue, and A. Rakotomamonjy. Svm and kernel methods matlab toolbox. Perception Systmes et Information, INSA de Rouen, Rouen, France, 2005.
[7] C. Carmeli, E. De Vito, and A. Toigo. Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem. Anal. Appl. (Singap.), 4(4):377–408, 2006.
[8] V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Comput. Surv., 41(3):1–58, 2009.
[9] H. W. Engl, M. Hanke, and A. Neubauer. Regularization of inverse problems, volume 375 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht, 1996.
[10] M. Hein, O. Bousquet, and B. Schlkopf. Maximal margin classification for metric spaces. Journal of Computer and System Sciences, 71(3):333–359, 10 2005.
[11] H. Hoffmann. Kernel pca for novelty detection. Pattern Recogn., 40(3):863–874, 2007.
[12] P Niyogi, S Smale, and S Weinberger. A topological view of unsupervised learning from noisy data. preprint, Jan 2008.
[13] R. Rifkin and R. Lippert. Notes on regularized least squares. Technical report, Massachusetts Institute of Technology, 2007.
[14] L. Rosasco, M. Belkin, and E. De Vito. On learning with integral operators. J. Mach. Learn. Res., 11:905–934, 2010.
[15] S Roweis and L Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, Jan 2000.
[16] B. Sch¨ lkopf, J. Giesen, and S. Spalinger. Kernel methods for implicit surface modeling. In o Advances in Neural Information Processing Systems 17, pages 1193–1200, Cambridge, MA, 2005. MIT Press.
[17] B. Sch¨ lkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson. Estimating the support o of a high-dimensional distribution. Neural Comput., 13(7):1443–1471, 2001.
[18] S. Smale and D.X. Zhou. Geometry of probability spaces. Constr. Approx., 30(3):311–323, 2009.
[19] I. Steinwart and A. Christmann. Support vector machines. Information Science and Statistics. Springer, New York, 2008.
[20] I. Steinwart, D. Hush, and C. Scovel. A classification framework for anomaly detection. J. Mach. Learn. Res., 6:211–232 (electronic), 2005.
[21] J. Tenenbaum, V. Silva, and J. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, Jan 2000.
[22] A. B. Tsybakov. On nonparametric estimation of density level sets. Ann. Statist., 25(3):948– 969, 1997.
[23] U. Von Luxburg. A tutorial on spectral clustering. Statistics and Computing, 17(4), 2007. 9